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ABSTRACT 


ate failure trends^^SuV^wth Cla ' 1 , Sp ^ Cec f aft were analyzed to evalu- 
s ho wed that the test prog rims were h ie hlv *Jf? Ct ' Vetle&s ° r tests ' The study 
of in-orbit reliability. There was onlv a sin^f CtlV f l ? en f urin 8 a hi g b level 
4^ years of in-orbit operation on 1 ? V«, V? catastrophic problem in 
that in-orbit failure rates are hiehlv ™ T u® results also indicate 

failure rates. The data suggest thLt tesVeffect* ?." d SyStems test 

used to guide the content of a , ctiveness estimates could be 

goals are achieved. Cost conside t0 ensu /‘ e that in-orbit reliability 
tive action program to achieve a near^r^f 1 W -1 St that an a K«rcssive correc- 
be implemented. " ° failure rate for all testing should 

margi Jfy'effectTvt tn^oV"’*,, 1 ’““"‘i* 1 ?” «,« f„ urld lo be 

tion is an effective quallficaticuftest l8 ^. e / lClen 1 C,es . - S i ,£terr - s icvel vibra- 
tor qualification than for acceMance tin ,1 f ‘' SU are more =«octive 
during qualification tests or during unit ac ~ rob ems are detected 
therm? 1 tests were mareinallv *• , ac, ~ e P tan ce, System acceptance 
test technique. Most thermal vacuumTe f r°', ably because °f deficiencies in 
60 hours of testing. test failures are detected in the first 

estimate the defeettV c popuTatio^and"^ US ^ unit and systems lev^I to 
performance. The in-orbit rurrml f P^ict m-orbit or systems test 

which implies a constant test effectivenets '^h^in ^rbft filope 

generally decrease with v ' In_ °^htt failure rates 

indicating reliability growth The^vsTem s t & Spacocraft series - thus 
failure rate data cannot be accuratelv ™ t-* j * nd ln ' orblt cumulative 
because the slopes are generally different ^ Dlia " e pJot 

testing. The t e s t ^ r o g r a^m s h o u kf b e ^ o ns r ri 1 ° & W f & ' 1 U F & d ° 00t perrnit reduced 
effective program is one £ which a nra l *™? . l "' ura,lCC - Tbe ™ost cost 
aggressive corrective action A rontinuirfo ^ f / ai . lure ratc 1S achieved through 
and improve the -cre.nl^fc^ tp —are 
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APPROACH 


All failure reports for three commercial spacecraft programs (16 
spacecraft) have been reviewed, categorized, and analyzed. These data 
include in-house Trouble and Failure Reports (TFRs), International Failure 
Reports (IFRs), and in-orbit problems as reported by customers. For 
simplicity, each event shall herein be referred to as a failure report, or 
FR , even though it should be noted that each FR does not necessarily repre- 
sent a failure, since anomalies, problems, intermittent?, wear-out, and 
failures were all reported on the FRs. 

A total of 1991 FRs and 90 orbit reports were corrected and coded. 
Each FR was described on an 80 column format with 27 discrete pieces 
information. Where missing or contradicting information existed, altern^ ve 
sources (e. g. , spacecraft logs) were utilized to correct and/or complete the 
data. The description of the information codes entered in each column is 
provided for reference in Appendix A. 

Each of the 2081 data points was then evaluated in a bulk data analysis. 
Various sorting and cross sorting provided information about overall test 
performance and specific problem areas. The test time to failure was 
estimated for each of the systems test and in-orbit FRs. The thermal 
vacuum test time to failure data were obtained directly from the respective 
spacecraft log books. 

The above data were used for the Duane analysis where the Systems 
Test and in-orbit Duane plots were calculated for each spacecraft. The linear 
regression coefficients were estimated, using the logs^Q of the cumulative 
failure rates and test times. The combined systems test and in-orbit cum- 
ulative failure rates were also plotted for individual spacecraft. 

The test effectiveness for specific test phases of initial-ambient, 
vibration, and thermal vacuum, at both unit and systems test levels, was 
evaluated In accordance with the Timmins Test Effectiveness Equation. The 
test effectiveness cf the entire unit and systems test program was also 
evaluated. In addition, the ability of the qualification and early production 
tests to detect design problems was analysed. The use of test effectiveness 
as a reliability prediction technique was developed. The attributes of a cost 
effective test plan were developed. 
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3. FAILURE DATA BASE 


3. 1 DATA BASE SUMMARY 


/u ilUr t S *t nd an ? malies whlch comprise the data base are 
Figure 3-1 scheduling of these programs is shown in 


fl _„ a Program 1 consisted of two separate contracts. The first (A) was 
?dH^ Pr ° °? P u. and four flight s P a «craft and the second (B) was for our 
,pa ? ecraf iv ° ,tha spacecraft .ucc.^fly 

J£e It e utilizL a ma a f ^ W ° Version of P ^gram 1, both in payload and 

rvice. Program 3 was a new contract utilizing the 


TABLE 3-1. PROGRAM SUMMARY 


Program 


Spacecraft Quantities 


Total for all 
pr tgrams 


Prototype + four flight 
Four flight 

Prototype + three flight 

Three flight + one payload 
shelf 

Two prototype 
Fourteen flight 
One payload shelf 


j In-Orbit j 

Number 

E Years 

4 

18.16 

3' 

10.52 

3 

9.99 

2** 

5.27 

12 

43.94 


Remarks 


Large communications payload 
Separate buy of same spacecraft 
Scaled down version of Program 1 
Follow-on contract to Program 2 


Average of 3-2/3 years per 
spacecraft 


*F 6 was launched but booster failed. 

1 *F-6 has not been launched, and the shelf has not been integrated to provide a spacecraft. 
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same basic spacecraft as Program 2, Therefore, the three flight spacecraft 
were considered equivalent to a follow-on buy and were sequentially num- 
bered, beginning with F-4. No prototype spacecraft was fabricated or 
tested for this program. A payload equipment shelf was additionally inte- 
grated with its respective units. Only two of the spacecraft have been 
launched, and the shelf has not been integrated into a spacecraft. Both of 
the launched spacecraft are in service. 

The data base for this study includes all failure reports (FRs) gener- 
ated on Program 1 from its contractual start date, through the last launch 
(F-l). All FRs to date on Programs 2 and 3 are included. All orbit anomally 
reports to date {March 1, 1977) by the respective customers are also included 
These reflect a total of almost 44 orbit years accumulated on the 12 space- 
craft in orbit. 

Because of the differences between programs in failure reporting 
methods at the lower levels of assembly, data were counted, but not 
analyzed, for FRs below unit level. For this study, a unit was defined as 
an identifiable (by serial number) piece, or pieces, of hardware which 
were formally acceptance tested either by a vendor or by Hughes Aircraft. 
These could and did include some major subassemblies which were 
delivered and then integrated to form a larger assembly (unit) and 
acceptance tested again prior to delivery to the spacecraft for integration. 

An example of this was the high power traveling wave tube amplifier (TWTA) 
unit. It consists of two major subassemblies: an electronic., power controller 
and the traveling wave tube. Each of these was acceptance tested and 
delivered by the respective vendors and then integrated to form the TWTA 
and acceptance tested again. Each of these was considered to be a unit 
for this study. Serial numbers of each were tracked by the configuration 
management system. Each subassembly could be removed as a unit and 
reintegrated to form a different unit as test and performance results 
required, 

To assess the relative complexity of each of the programs, Table 3-2 
summarizes both th- types and quantities of units which comprised a given 
flight spacecraft, /he data are divided into eight different subsystems for 
each spacecraft and the number of mechanical units is also identified for each 
subsystem. A normalization factor was calculated, based on number of 
units per spacecraft, by using Program 1 as the reference. Another way to 
compare the complexity of programs is through the use of electronics parts 
count. Table 3-3 presents this data. It was concluded that values of 2. 90 
for Program 2 and 2. 44 for Program 3 should be used when data required 
normalization for program to program comparisons. This was derived by 
use of a weighted average using 3 for electronics parts complexity, 2 for 
total units, and 1 for types of units. 

All FRs were divided into five distinct groups by hardware develop- 
ment phase: subunit, unit, systems test, orbit, or not applicable. The 
distinction between subunits and units has been previously discussed. 

Systems test level was defined to begin at delivery of units to the spacecraft 
and continue through launch of each spacecraft, including those tests per- 
formed at launch base. After review of each FR, a small number were 
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TABLE 3-2. UNIT TYPES AND QUANTITIES 


Unit Types 

Program 1 

Program 2 

Program 3 

Total unit types per spacecraft subsystem 




Communications 

42 

8( 2)< 1 > 

8( 2)< 1 > 

Antenna 

12(11) 

5( 5) 

5( 1) 

Telemetry and command 

35( 9) 

3( 1) 

3( 1) 

Propulsion 

10( 4) 

5( 1) 

5( 1) 

Power 

13 

6 

6 

Structure 

22 ( 6) 

15( 3) 

17( 3) 

Attitude control 

7( 1) 

8( 3) 

9( 3) 

Apogee kick motor 

3< 1) 

2( 1) 

2( 1) 

Total 

144(33) 

82(16) 

55(16) 

Total units per spacecraft subsystem 


mam 


Communications 

246 



Antenna 

18(17) 

ni 

5( 5) 

Telemetry and command 

62(10) 

4( 1) 

4( 1) 

Propulsion 

75(18)<3> 

10( 2) 

10( 2) 

Power 

26 

21 

21 

Structural 

26(11) 

43( 3)< 4 > 

46( 3)< 4 > 

Attitude control 

2( 3) 

12{ 5) 

13( 5) 

Apogee kick motor 

3( 1) 

2( 1) 

2( 1) 

Total 

468(60) 

150(18) 

154(18) 

Normalization factor by unit types 

1 (reference] 

2.77 

2.62 

Normalization factor by total units 

1 (reference) 

3.12 

3.04 


Notes: ( ) * Number of mechanical units with no electronics or electrical hardware. 

(1) Does not include 17 receiver unit parts that were included (tracked) on 
Program 1 

(2| Because of (1), 25 receiver units were not included 

(3) Includes 42 tracked heater elements 

(4) Includes 28 tracked temperature sensors 


TABLE 3-3. ELECTRONIC PARTS COUNT PER SPACECRAFT 


Number of electronic parts 

Program 1 

Program 2 

Program 3 

16,846 

8,370 

8,569 

Normalization factor 

1 (reference) 

2.01 

1.07 


3 - 



































excluded from the data ba9e for one of the following reasons: 1) it was not 

written against spacecraft hardware (e. g. , test equipment malfunction 
which had no effect on spacecraft hardware or the test program), 2 ) it was 
a duplicate of another FR documenting the same problem but originating 
in a different area, and/or 3) the original FR or copies of the FR could not 
be found in the files -to- enable-coding . 

A summary of total data base is presented in Table 3-4. The FRs 
which form a basis for the analytical study are shown within the heavy line 
box. Table 3-5 presents the same data as a percentage of the whole, by 
program. Of interest is the increasing effectiveness of the subunit level 
tests on each program with the systems test level decreasing and the unit 
test level remaining relatively constant. This is probably the result of 
both varying FR reporting criteria on each program and a learning process 
whereby the subunit testing became more effective in screening problems. 
The fact that so few FRs were present on Program 3 at systems test level 


TABLE 34. NUMERICAL SUMMARY OF 8ULK DATA 


Program 

Number of Spacecraft 

Total FRs 
Reviewed 

Level of FRs 

Not' 

Applicable 

Subunit 

Unit 

Systems 

Test 

In-Orbit 

1 

8 + prototype 

1,426 

286 

721 

263 

68 

88 

2 

3 + prototype 

355 

104 

177 

42 

13 

19 

3 

3 + shelf 

300 

98 

160 

18 

9 

15 

Total 

14 flight + 2 prototype + 1 shelf 

2,081 

488 

1,058 

3" 

90 

122 


'Includes duplicates, not spacecraft hardware problems, etc. 


TABLE 3 5 PERCENTAGE SUMMARY OF 8ULK DATA 


*1 

Program 

Number of Spacecraft 

Total FRs 
Reviewed 

Level of FRs 

Not* 

Applicable 

Subunit 

Unit 

Sys terns 
Test 

In-Orbit 

i 

8 + prototype 

100 

20.0 

50.6 

18.4 

4.8 

0.2 

2 

3 + prototype 

too 

29.3 

40.9 

118 

3.7 

5.4 

3 

3 + shelf 

100 

32.7 

53.3 

6.0 

30 

5.0 

Average 

14 flight t2 prototype + 1 shelf 

100 

23.5 

50.8 

15.6 

4 3 

59 


Includes duplicates, not spacecraft hardware problems, etc. 
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(18, and 6 of these were induced) requires further investigation The h 

As discussed, many of the units utilized o-> «rh *u„ «. u 
grams were supplied to Hiehes Aim o„ each of the three pro- 

£2Ji ZiSz 

j*v.£r* p *V th * «»& i.“ i 

* ~ .r h ° d * - >»™*. 


TABLE 3-6. FAILURE REPORT RATES COMPARISON 


“fograrn 

Number of 
Spacecraft 

Total Unit 
Systems, and 
In-Orbit FRs 

Electronic Parts 
per FR 

FR Average 
per Spacecraft 

Normalization 

Factor 

Normalized FR 
Average per 
Spacecraft 

1 

8 + prototype ' 

1,140 

126.6 

134.1 

1 (reference) 

134.1 

2 

3 + prototype * 

251 

118.7 

71.7 

250 

179.25 

3 

3 + shelf* 

202 

148.4 

57.7 

2.44 

140.79 

Combined 

-15.5 

1,593 

127.1 

102.7 

1.67 

171.51 


‘Each of these * '/, of a spacecraft 


TABLE 3-7. HUGHES VERSUS INTERNATIONAL 
FAILURE REPORTS 



Unit Failure Reports' 


Program 

Hughes 

International 

Total 

1 

348(48) 

373(52) 

721(100) 

2 

63(36) 

114(64) 

177(100) 

3 

16(10) 

144(90) 

160(100) 

Total 

427(40) 

63) (60) 

1,056(1001 

'Percentage shown in parentheses 
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3.2 FAILURE REPORT CATEGORIES 

Two basic breakdowns of the FRs were required to effectively 
present and analyze the data. A distinction between FRs on the qualification 
(prototype) and acceptance (flight) spacecraft was made to distinguish between 
the- different test programs. The second breakdown was one of "importance 1 ' 
for each of the two data sets. A significant number (about 20 percent) of all 
the FRs "integration and/or test" (I&T) induced. These were not the fault 
of the hardware, but were the result of human errors and required unit or 
system rework. Some (about 11 percent) of all the FRs were secondary in 
nature, in that although FRs were initiated, no resultant physical hardware 
action was taken. A third distinction (about 1 percent) was made for FRs 
initiated against hardware for an anomaly which could not be repeated. All 
remaining FRs (about 63 percent) were classified as primary and were 
included in the analysis data base. A summary of these several categories 
is presented in Tables 3-8, 3-9, and 3-10 for international units, Hughes 
produced units, and systems level test FRs, respectively, for both quali- 
fication and acceptance hardware. 

Essentially, all qualification hardware was produced at Hughes for 
the two prototype spacecraft. Tables 3-8 and 3-9 show that a significantly 
.higher number of secondary FRs were generated internationally; 64 percent 
were for out-of-specification conditions with no retest/rework required. 

This is probably a result of the formal contractual interface between Hughes 
and these subcontractors with regard to out-of-specification conditions. 

Beth Hughes and international unit test results showed that electrical 
test errors caused about 41 percent of the IS-T induced FRs, and electrical 
test overstress caused another 24 percent. Hardware mishandling accounted 
for an additional 12 percent. It is probable that increased emphasis on lest 
procedures and safety practices, both electrical and mechanical, would have 
been cost effective. 

System test FRs are summarized in Table 3-10, A total of 31 per- 
cent of all FRs on Programs i and 2 were against the prototype hardware. 

The I&T induced failure ’■ate remained about constant at 23 percent on all 
programs 'or prototype or flight hardware. The susceptibility of Program I 
to propulsion subsystem line heater damage is apparent. Hardware handling 
problems and electrical test errors were significant on all programs. The 
small number of primary FRs at the systems level un Programs 2 and 3 
makes significant analysis difficult. However, the fact that the sample 
size is so small is a credit to the effectiveness of the subunit and unit test 
programs. The reduction in secondary out-of-specification and test error 
problems on D rograms 2 and 3, relative to Program 1, probably reflects 
mare mature systems test procedures and specifications requirements as 
a result of experience gained on the first program. It should he noted that 
the systems test teams were derived from the same organization on all three 
programs. 
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TABLE 3-8. FAILURE REPORT CATEGORIES • INTERNATIONAL UNITS 



Program * 

Sum 
of all 
Programs 

FR Type 

i 

2 

3 


Qualification (Acceptance 

Qualification 

Acceptance 

Acceptance 

Totals 

373 

114 

144 

831 

Quell f Scat ion/accep tance 
breakdown 

2(1 ) 

371(99) 

2(2) 

112(98) 

144 

... 

Less secondary 

0 

25(7) 

0 

3b!31) 

42(29) 

102(16) 

Less l&T induced 

1(50) 

62(17) 

0 

12111) 

25(17) 

100(16) 

Less anomalies 

0 

0 

0 

0 

0 

0 

Total primary 

1(50) 

284(77) 

2000) 

65158) 

77(541 

429(68) 

Secondary reasons 







Out of soec/OK 


20 


17 

28 

65(64) 

Meas teelWNo retest 


0 


7 

4 

11(111 

Test equip/No retest 


1 


S 

1 

7(7) 

Mfg error/multiples 


1 


0 

0 

1(1) 

Test error/No rework 


3 


6 

8 

18(18) 

Aromalies/No retest 


0 


□ 

0 

-0 

Misc/No retest 


0 


0 

0 

0 

Total secondary 

0 

25 

0 

35 

42 ■ 

102(1001 

l&T induced reasons 







Damaged wires, heaters, 
etc. 


7 


1 

1 

919) 

Elec test overstress 


17 


2 

7 

26126) 

Elec test error 


30 


4 

9 

43143) 

Hardware handling 

1 

5 


2 

2 

10(10) 

Wrong environment 
imposed 





6 

10(10) 





Bad part selection in 
test 


2 


O 

O 

2(2) 

RCS test error 


0 


O 

0 

0 

Test f:- v >uie induced 


0 


0 

0 

0 

Total I3 iT induced 

1 

62 

0 

12 

25 

100(100) 


Percentage shown in Oorenthesei ; the data have not been normalieed 



TABLE 3-9. FAILURE REPORT CATEGORIES • HUGHES UNITS 


FR Type 

Program* 


1 

L ? 

3 

Sum 

Totals 

Qualification (Acceptance 
34R 

Qualification 

Acceptance 

Acceptance 

Programs 

Qual if icat \ on /accept 
breakdown 

411121 

307(88) 

29(46) 

34(64) 

16 

16 

427 

Lass secondary 

0 

5(2) 

0 

5115) 

8(501 

18(4) 

Less l&T induced 

9(221 

72(23) 

8(28) 

11(32) 

4(251 

104(24) 

Less anomalies 

0 

0 

0 

0 

0 

0 

Total primary 

32(78) 

230(75) 

21(72) 

18(53) 

4(25) 

305(72) 

Secondary reasons 






' 

Out of spec/OK 
Maas, tech /No retest 
Testequip/No tetest 
Mfg error /multiples 
Test error/No rework 


1 


3 

1 

7 

4(22) 

0 

0 

7(39) 

1(6) 

4(22) 

Anomaltes/No retest 


2 


2 


Misc/No retest 


2 




2(111 

Total Secondary 

0 

5 

0 

5 

8 

18(100) 

l&T induced reasons 






h - 

Damaged wires, hearers, 
etc. 

4 

8 

0 

5 

0 

17(16) 

Elec test overstress 

0 

18 

4 

1 

0 

23(22) 

Elec test error 

1 

34 

1 

1 

3 

40(39) 

Hardware handling 

4 

S 

2 

3 

1 

16(15) 

Wrong environment 

0 

0 

0 

o 



imposed 







Bad part selection 
in test 

0 

0 

0 

0 

0 

0 

RCS test error 

0 

s 

1 

0 

0 

7(7) 

Test fixture induced 

0 

0 

0 

1 

0 

mi 

Tola' l&T induced i 

9 

72 

8 

M 

— 

. _ 

4 

104(100! 


■Percentage shown in parentheses, the data have-not-beerv-narmalived 
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TABLE 3-10. FAILURE REPQRTCATEGORIES - SYSTEM TEST 


FR Type 

Program* 


! 

I 2 

3 

Sum 


Qualification 

Acceptance 

Qualification 

Acceptance 

Acceptance 

Programs 

Totals 

263 

42 

18 

323 

Qua! i f icat ion /accep t 
breakdown 

86(32) 

178(58) 

12(29) 

30(71) 

18 

... 

Less secondary 

8(7) 

2001) 

0 

1(3) 

2011 

2919) 

Less l&T induced 

19122) 

38(21) 

3(25) 

8(27) 

6(33) 

74(23) 

Less anomalies 

0 

10(6) 

0 

0 

0 

1013) 

Total primary 

60171) 

110(62) 

9(76) 

21 (70) 

101561 

210165) 

Secondary reasons 







Out of spec /OK 

0 

11 




1 

Meas. tech/No retest 

0 

0 




o 

Test eqnip/No retest 

0 

0 





Mfg erroi/inultiple5 

0 

□ 



2 

217) 

Test error/No rework 

2 

8 




10(34) 

Anomalies/No retest 

0 

0 


1 


1(4) 

Misc/No retest 

4 

1 




5(17) 

Total Secondary 

6 

20 

0 

1 

2 

29000) 

l&T induced reasons 







Damaged wires, heaters, 
etc. 

12 

13 

0 

1 

2 

23(33) 

Efac test oversttets 

0 

2 

0 

2 

1 

5(7) 

Elec test error 

4 

9 

2 

1 

2 

18(24) 

20(271 

Hardware handling 

3 

13 

1 

2 

i 

Wrong environment 
imposed 

0 

1 

0 

0 

0 

1(1) 

8ad part selection in 
test 

0 

0 

0 

0 

0 

0 

RCS test error 

0 

0 

0 

0 

0 

0 

Test fixture induced 

0 

0 

0 

2 

0 

2(3) 

Total t&T induced 

-J1 [ 

38 

3 

8 

6 

74(100) 


'Percentage shown in parentheses; the Polo have not been normaliaed 
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3.3 TEST SCREEN EFFECTIVITY 

origin ?° r i ed by te3t acfcivit y to determine the 

origin oi each b R. This sort reveals the screening effectivitv of each 

in-orhit^esX Thes^d ^ dcv ° lo P ment - Whe " combined with the 

anaWtical m^Vhn i become the basis for the evaluation of the two 

nalytical methods presented in Sections 4 and 5 of this report. 

rate, of 17 6 20 ^ a „d n" 0 °° ” . , = s ' rstems level acceptance FR 

^ ^ -zsr 

Table. f*n l . C !2*5' T, Vir, ; r ‘ m '"! s within a “ S1 are discernible from 

lames 3*11 and 3-12. For all vibration tests, Z (longitudinal! and >: n a f 

SPF ^ br ;. 

comp.lt” of "ellc^on S cVwest^ ?' " d ''P“" 
eclipse condition and was S erefore S£ E tT' 
environment indicated no strong pref“as S'^rV™™’^,.'" 1 

spacecraft 6 alVi^nm Jnt^to* ' 1 ^ nd 3 ~ 12 "ferc grouped and distributed by 

spacecraft assignment to understand the significance of the test seteen 

“? “ l° n “ > ‘“a' 001 ' Cot trencls - Thi ‘ is presented on Ftp, res 3.2 and 

for uMtt'rL'^s wer S e VS m’it 1 '? 1 *'* *™“" Th « fo “" « roupingsVholen 

tests, 2| vibration and post vibrati“Ter^ 

ZTnV^ZV R F i S JlZl S : ,T 0r : h6 , ^'4 >-* test Vcrcens, 
integrated systems test, (1ST), 2) vibration a nd “sPT "jT aU ' Tv'tcJtV \°DCTV/ 

carr_.d as a separate spacecraft grouping to enable analysis. 
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TABLE 3-11. QUALIFICATION TEST SCREEN - NUMERIC 



Program 1 

Program 2 

Mm 


Teat 

Interna- 

Hughes 

Systems 

Interna- 

Hughes 

Systems 



Screen 

tionaf Unitt 

Units 

Level 

tfonal Units 

Units 

Level 

Total 

Initial inspection 



7 


a 


9 

Mechanical mfg. 


3 

3 




6 

Initial elec, perl, test 

« 


5 

2 

4 

1 

24 

Perf. test - 

not identified 


4 



6 


10 

Final alec, perf 

lest 





a 


2 



Transition 

^ 


1 




1 

STWESTV 

Hot 










Cold (eclipse) 






1 

1 

Initial 1ST 





11 



1 

12 



Setup 



1 




i 



Not defined 


3 

2 


i 


6 

Vibration 


X-axis 


1 

6 


i 


8 



Y- axis 







2 



2-axis 


1 



i 


6 

SPT (post vibration) 


a 



i 

6 

18 


Hot 


i 





1 

TV/STV 

Cold 



5 




5 


Eclipse simulation 



1 




1 


Ukn or transition 



1 




1 


Unknown 



. 





Thermal 

Transition 





3 


3 

cycle 

Hot 



. 






. Cold 


4 





a 

Final 1ST 



- 

- 

1 

e 

. 


— 

Final inspection 


1 




1 

tiHK 

ACS tests 




1 





i 

Subtotal ' 

) n.o) 

32 (34.4) 

80 (64.5) 

2 (6.31 

21 (65.6) 

9(28.1) 

126 

Total 

93 


3a 

I 

125 


•Percentage shown in parentheses; the data have not been normalized 
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TABLE 3-12. ACCEPTANCE TEST SCREEN • NUMERIC 


Initial inspection 
Mechanic* mlg. 

Initial dec. purl. test 
Pert, test not jdentif 
Final eitc. perf. test 

DCTV/ESTV 

Initial 1ST 

Vibration 
Unit ranclom 
Spacecraft-sine 
and random 

SPT l Post vibration) 
TV/ST V 

Thermal cycle 


Final 1ST 
Final inspection 
RCS tests 
Launch opera] ions 

Subtotals* 


f Transition 
[ Hot 

Cold (eclipse! 

Setup not defined 

X-axis 

Y-axis 

2-axis 

Hot 

Cold 

Eclipse simulation 

Unk. or transition 

Unknown 

Tf am it ion 

Hot 

Cold 


Program I 


Hughes Systems 
Units Level 


Program 7 


'Percentage shown mi parentheses, the data Have not been nonnah/ctf 


Hughes 

Units 

Systems 

Level 

1 

1 

1 

4 

3 

e 



2 

< 

4 


4 

1 

1 

i 


4 


2 

2 


IB 

21 

(17.3) 

(20.2) 


Program 3 


















Figures 3-2 and 3-3 show a number of significant features about the 
primary data base. Program 1 was generally more uniformly distributed 
within each test group than were Programs 2 or 3. This is probably the 
result of the larger sample size. At the unit level, at least 42 percent of 
all FRs were found in group i. -- initial assembly and performance check- 
out, Some of group 4 (undefined performance test) undoubtedly belong in 
group 1; therefore, the initial assembly and performance test is the most 
important phase for test screening at the unit level. Vibration (group 2) 
is less effective (15 percent) than thermal (21 percent) tests (group 3) on all 
programs and about 21 percent of all unit FRs were in group 4 -- undefined 
performance. Within the unit groupings, the qualification data are not sig- 
nificantly different from acceptance. Qualification vibration was somewhat 
more effective than acceptance (21 percent of all qualification FRs versus 
17. 5 percent of all Program 1 FRs and 17 percent versus 8 percent for all 
Program 2). Thermal testing was somewhat less effective at qualification 
(18 percent versus 21 percent for Program 1). However, this difference was 
very pronounced at the systems level. Almost 42 percent (Program 1) and 
55 percent (Program 2) of the respective prototype spacecraft FRs occurred 
in vibration. This contrasts to 16 and 20 percent overall and only 3 and 
10 percent without prototype spacecraft FRs. Therefore, vibration and 
SPT were very import nt for systems level qualification (prototype spa ce- 
craft) but were significantly less effective screens for systems level 
acceptance (flight spacecraft). Systems level TV tests were more effective 
for the flight spacecraft than for the prototype spacecraft. 

Figure 3-2 shows that as the unit tests screens mature with time, the 
effectiveness of the initial ambient tests remains constant at about 41 per- 
cent. Unit vibration generally trends downward with each successive 
spacecraft and program, and thermal test grouping trends upward in 
importance. The percentage of undefined unit performance tests (group 4) 
trends upward (19 percent on Program 1 to 27 percent on Program 3), 
indicating that attention to detail on the FRs waned with each successive 
program. It should also be noted from Table 3-7 that the percentage of 
foreign participation also increased with each successive program. The 
high percentage of group 4 for the F-4 and F-5 assigned units from Pro- 
gram 3 makes the Program 3 evaluation difficult. By comparison with 
Program 3 F-6 assigned units, where a high percentage is assigned to 
group 1 (59 percent) with a corresponding low group 4, one could conclude 
that the vast majority of group 4 Program 3 FRs actually occurred during 
initial checkout and the 44 percent overall group 1 value is very conservative. 

At the systems level the initial assembly integration and 1ST (group 1) 
tests are again very effective: 50, 33, and 50 percent for Programs 1, 2, 
and 3, respectively, and 48 percent overall. 

The systems level sine and random spacecraft vibration tests 
produced only 1 FR during acceptance (Z sine), and five FRs were additionally 
reported after the vibrational environment during the Systems Performance 
Test (SPT), 
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PERCENT OF WHOLE BY SPACECRAFT AND PROGRAM 



FIGURE 3-2. DISTRIBUTION OF UNIT TEST SCREENS 
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HUME 

R'CAL VALUES 



INITIAL 

SERF 

VISITATION 

ANoirr 

TV/TC 
* FINAL 
FERF 

UNKN 
FERF 
+ RCS 

l TOTALS 

QUALIFICATION 

NOT 

AMlONEO HI 

F-l 

F-2 

F-3 

FA 

F-S 

M 

F-7 

F-8 

IS 

IS 

as 

37 

H 

« 

as 

ii 

i* 

33 

7 

4 

71 

11 

10 

9 

0 

12 

4 

13 

f 

7 

14 

ie 

ii 

n 

14 

10 

11 

16 

6 

16 

14 

10 

17 

tfl 

7 

Q 

? 

e 

33 

42 

77 

74 

62 

C3 

62 

41 

36 

67 

TOTALS 

228 

as 

116 

108 

S47 


PnOaRAM 2ANO 1 UNIT TESTS 



QUALIFICATION 

a 

4 

6 

6 

23 

NOT 

ASSIGNED (2) 

e 

0 

3 

0 

14 

F.1 

12 

3 

2 

6 

26 

F*2 

U 

1 

1 

3 

19 

F*3 

9 

2 

9 

6 

26 

NOT 

ASSIGNED 01 

1 

0 

0 

0 

1 

FA 

4 

3 

a 

7 

22 

F-6 

3 

1 

4 

11 

19 

F* 

13 

- 0 

6 

3 

22 

SHELF 

14 

» 

2 

0 

17 

TOTALS 

94 

IS 

40 

« 

187 


PROGRAM 1 
PROGRAM 2 
PROGRAM 3 

228 

48 

33 

96 

10 

I 6 

115 I 
20 
20 

106 

27 

1 21 

r "" 
108 

i «' 

totals 

312 

^ 111 

156 

| 166 

; 7»* 
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g^MS TEST-SCREENS 



J!u 

lERit/kL vA.ll 

ES “ 1 

INITIAL 
ASSEMBLY 
AND (ST 

VIBRATION 
AN0 8PT 

OCTV/fSTV. 
TV AND 
1ST 

ALL 

launch 

OPERATIONS 

TOTALS 

PROTOTYPE 

36 

25 

» 

NIA 

00 

F-1 

20 

1 

e 

I11F-2I 

36 

P-2 

3 

0 

4 

HF-1J 

8 

F-3 

11 

1 

2 

1 


F-4 

e 

0 

0 

0 

6 

F-5 

3 

0 

S 

7 

10 

F-0 

a 

1 

& 

1 

IB 

P-7 

4 

0 

2 

2 

a 

F-a 

6 

0 

4 

1 

10 

totals 

66 

28 

n 

16 

170 


PROTOTYPE 

2 

S 

2 

N/A 

9 

f t 

5 

t 

7 

0 

13 

P-2 

1 

0 

1 

0 

2 

F-3 

2 

0 

4 

0 

6 

F-4 

0 

7 

1 

0 

3 

F*S 

3 

0 

0 

0 

3 

F-6 

0 

0 

2 

tit A 

2 

SHELF 

2 

N/A 

N/A 

N/A 

2 

TOTALS 

15 

s 

L '» 

0 

12 


PROGRAM 1 
PROGRAM 2 
PROGRAM 3 

» 

10 

& 

28 

5 

2 

17 

14 

3 

16 

0 

0 

1 70 
XI 
10 

TOTALS 

101 

» 

64 

IS 210 1 



i'^gs blank 


i 


NOT IT- S'* 





















Figure 3-3 shows that an increase in Program 2 systems level TV 
occurred with a corresponding decrease in the initial (group 1) screening. 
The data shows that this may have been caused by both inadequate unit 
thermal testing (Figure 3-2 Group 3) and systems level initial performance 
testing (Group 1 of Figure 3-3). 

Figure 3-3 also shows that all of the launch operations FRs occurred 
on Program 1, The first spacecraft launched on Program 1 (the F-2) had 
1 1 FRs generated during launch operations. Although FRs continued to be 
generated against succeeding spacecraft, they accrued at an average of just 
over 1 per spacecraft. During the five launch operations of Programs 2 and 
3 no FRs occurred. This is probably the result of a mature test program 
with proven procedures. It should be noted that the F-l spacecraft of 
Program 1 was unique in that after the spacecraft systems level tests were 
completed it was stored and not launched. After all other spacecraft on 
Program 1 were tested, F-l was returned to Hughes, upgraded with the 
latest configurations (a few design changes) and then again put .'.hrough a 
systems level spacecraft test program prior to shipment to the launch base 
for launch. 

Table 3-13 was prepared, using the data for the systems level 
acceptance test screens of Table 3-12. It distributes the FRs by spacecraft 
sets of all, initial (F-l), all remaining, and F-I retest for Program 1 into 
three new groupings: 1) pre- spacecraft environment; 2) spacecraft environ- 

ment (vibration and TV); and 3) launch operations. As can be seen from 


TABLE 3-13. SPACECRAFT SYSTEMS LEVEL TEST SCREENS BY SPACECRAFT 







Program^ 








1 





3 1 

1 

Group 

SvsteTis Level Acceptance Test 
Screen by Spacecraft 

Total, 

All 

Space- 

craft 

F-1 

Space- 

craft 

Original 

F-2 

through 

F-8 

Space- 

craft 

F-1 

Space- 

craft 

Retest 

Total, 

All 

Space- 

craft 

F-1 

Space- 

craft 

F-2 and 
F-3 
Space- 
craft 

1 

Totals, j 
All | 

Space- 
craft 


Iniilal electrical and mechanical 
assembly and performance 

33 

3 

21 


4 

4 

0 

2* 

i 

OCT WE ST V 

5 


5 


6 

3 

3 

0 


Initial 1ST 

27 


19 

L- 

2 

4 

1 

3 

1 


Vibration 

1 


wm 

m 

0 

0 

0 

0 


SPT 

2 


H 


t 

1 

0 

2 

2 

TWSTV 

g 

■ 

8 

i 

0 

0 

0 

2 


Final 1ST and inspection 

14 

3 

9 

2 

1 

6 

4 

2 

1 

3 

Launch operations 

19 

If * 

8** 

N/A 

0 

0 

0 

0* * ‘ 


Total 

no 

31 

72 

8 

21 

13 

8 

8 


Average per spacecraft 


31 

10.3 





13 

4.0 

2 7 


•Shelf data not included (two FRs} 

••F ? wjs 1st sp.icecf.ift Munched, therel ore, F 7 Uni nr h operations FRs were a*b*tf 4 triiv .istiflnecl to F 1 .mrl 


vice versa 

• ««F 6 spdcecralt h.is not d$ yet been tetted 
have not been normalized. 
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such a grouping and distribution, there is * .... 

observations after the initial snaeo a. . a S 1 g n *.icant distinction in FR 
note that Program 3 tW ° test programs- 

spacecraft had three times as manv nrobl ° n l ° Pro S rar « 2 . Both F-l 
spacecraft. The F - 1 spacecraft' retest So bl S 35 a ^rage of succeeding 
unique evidence that the initial r i 1 problems of Program ] provide § 
efl.ctiix, • F -' «e.t program waa^ot s to.ally 


3.4 DEFECTS ANALYSIS 

The S r |«.gorie, aocordlog to th. 

/ he results of this division bv level of* uV** 1 *"* and 4 ) unknown. 

£f “££ .hown on F? eu «r 3 S ra 'd y ft 

evel FRf, are shown as a function of and 3 - 5 . The unit and systems 

occurred on a unit which was not eventilnV 18 "^ spacecra ^. Wher/a« FR 
was classified as "not assigned ‘ , l V”* ned to a spacecraft, it 
Hughes unit FR S * \ 7 he relationship c' ^ ■ ■ - J 


u., , ““ “ui assigned. 

“fjjj x Rs was maintained 
anomaly FR S were also added 


. , , , Al . 

Tr,«- -----'"■ship of international and 

Integration and test (I&T) FR 3 and 


in the numbers of FRsTccurHngTring^ 6 ? fT ° m early to late spacecraft 
program. An estimate of this trend 1 8 Siven spacecraft acceptance test 
the averages for the first set of spacecraft Tndfh each ^ Ure wherein 
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time * reSultin B - ‘ a te P spacecraft a^i^nt.^ cycle later in 

FR ratet! ‘in general! wo rkman^hk,' d & h exhibited relatively low 

craft ?nr e x? t0 the P rece ding spacecraft a11 low ” 

craft (F-3), No viable explanation lF-I) and the following space- 

5 »k »^SSS %• -ssf 

stated earlier, the F-2 spacecraft wL ft ff S r V? le F ' Z 3 P a cecraft). As 
abnormal amount of difficulty durino fh ^ laimchcd and had an 

were because of the "first ^ operations, most of which 

in the form a f ft o q u Z fy c h I7t « 3 " Y Wn redrawn on Fm ar „ 3 . 6 

spacecraft relationships can b ' hi " T/ ^ ^ & 

sm rc3pec / ivclvr * Thc not assigned FRs hive L™ 8 ™’” 1 and Programs 2 and 
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NUMBER OF FAILURE REPORTS BY TYPE 
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PROGRAM 1 
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FIGURE 3 4. PROGRAM 1 - FAILURE REPORT CAUSE BY SPACECRAFT 
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FIGURE 3 4 {CONTINUED). PROGRAM 1 - FAILURE REPORT CAUSE BY SPACECRAFT 
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NUM8ER OF FAILURE RETORTS BY TYPE 
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FIGURE 3-5 (CONTINUED). PROGRAMS 2 AND 3 
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The apparent reduced effectiveness of the qualification test program 
is due in part to the fact that the equipment complement was only approxi- 
mately 50 percent of a flight spacecraft. However, it is evident that the 
Program 1 systems test environment was significantly more effective as a 
qualification test screen than the unit test environment. The data also 
indicates that the design defects were not adequately screened during quali- 
fication environments, and significant numbers of design problems were 
revealed in subsequent flight spacecraft acceptance tests. 

As might be expected, workmanship FRs are the largest single 
category in the data base at both test levels for all three programs and for 
almost every flight spacecraft. The shifting of problems to the last space- 
craft in any set is clearly evident for workmanship FRs (see F-4 and F-8 
spacecraft for Program 1, F-3 spacecraft for Program 2, and F-6 space- 
craft for Program 3). Overall, the trend toward reduced numbers of FRs 
with each progressive spacecraft is consistent, with the exception of the 
workmanship FRs which trend upward for the last spacecraft in each set. 

The second most important category of acceptance FRs was the I&T 
induced group. These FRs were significantly high for each spacecraft. For 
the later spacecraft at the systems level there were almost as many I&T 
induced failures and problems as the test program revealed in the space- 
craft hardware. 

Defective part problems occurred throughout the acceptance test 
programs and did not appear to be affected by spacecraft number or test 
maturity. 


Design FRs followed the expected downward trend, but not to the degree 
one would expect. The initial spacecraft of each program revealed substantial 
numbers of design defects (despite the qualification program) which decayed 
to a relatively constant rate for Program 1 from the 4th to the 8th spacecraft. 
The F-4 spacecraft did involve some new designs on Program 3; this resulted 
in a new start of this design FR decay process. 

The similarity of data from program to program was verified by the 
use of the normalisation factors presented above. When the primary data 
is sorted and divided by the number of spacecraft within the set as appro- 
priate, and then normalized, the results are as shown on Tables 3- 14 and 
3-15 for qualification and acceptance. Program 1 qualification FRs are 
similar to Program 2. The same is true of the FR per spacecraft average 
of the first flight spacecraft set for Programs 1 and 2. The last four space- 
craft on Program 1 {set ID) was almost identical in FRs per spacecraft to 
Program 3. In summary, the data, although derived from three different 
programs, is consistent when normalized for either qualification or 
acceptance. Therefore, it was concluded that program and level of develop- 
ment distinctions are not necessary for FR defect analysis purposes. 

Tabic 3-16 shows a summary of all primary FRs by cause. Design 
FRs accounted for aLmost 50 percent of all qualification FRs, but only 
16. 7 percent of the acceptance group. Workmanship accounted for 51.4 per- 
cent of the acceptance FRs and 35, 2 percent of the qualification group, 
Summaries segregated by the cause of failure are shown in Tables 3-17, 

3-IS, and 3- 1 9 for workmanship, part, and design defect types, respectively. 
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TABLE 3-14. NORMALIZED PRIMARY QUALIFICATION 
DATA BASE BY SPACECRAFT 



Program 

1 

2 

Normalization factor 

1 (ref.) 

2.50 

Raw data: 



Unit 

33 

23 

Systems level 

60 

9 

Normalized data: 



Unit 

33 

57.5 

Systems level 

60 

22.5 

Total 

93 

79.0 


TABLE 3-15. NORMALIZED PRIMARY ACCEPTANCE DATA BASE BY SPACECRAFT 



Program 


1 A 

18 

2 

3 

Number of spacecrcft 

4 

4 

3 

Unit: 3’/,* 

System: 3 

Normalization factor 

1 (ref) 

1 (ref) 

2.50 

2.44 

Raw data/spacecraft: 
Unit 

Systems level 

76.5 

16.75 

520 

10.75 

27.67 

7.0 

23.14 

2.67 

Normalized data/spaceeraft: 
Unit 

Systems level 

76.5 

16.75 

52.0 

10.76 

69.18 

17.50 

56.46 

6.51 

Total 

93,25 

62.75 

86.68 

62.97 


'Shelf units = 14 of a spacecraft 


TA8LE 316. ALL PRIMARY FAILURE REPORTS BY CAUSE 


FR 

Type 

Qualification 

Acceptance 

Quantity 

Percent of 
Total 

Quantity 

Percent of 
Total 

Workmanship 

44 

35.2 

421 

51.4 

Part 

16 

12.B 

238 

29.1 

Design 

62 

49.6 

137 

16.7 

Unknown 

3 

2.4 

23 

2.8 

Total 

125 

- 

819 

- 


U28 
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Table 3-17 dramatically shows that 60 percent of all workmanship 
FRs occurred during installation and/or assembly for both qualification and 
acceptance tests. No 1 ’learning curve" improvements are apparent. Con- 
centrated effort in this single area may have had a significant impact for 
both cost and schedule of the programs. Bad electrical connections (welds 
and solder joints) accounted for almost 16 percent of the acceptance work- 
manship problems. 

Manufacturing process was the largest part FR problem; 43. 8 per- 
cent for qualification and 30. 7 percent for acceptance (possibly higher 
because of the unknowns 39. 9 percent). Part internal contamination was 
also important (17. 2 percent of part FRs) for acceptance. In general, the 
process problem was significant throughout ail three programs. 

The summary of design FRs by cause is shown in Table 3 - 19 . Five 
different types of problems were almost equally important to the qualifica- 
tion program. Circuit design and mechanical installation were followed by 
manufacturing process errors and innate physical property problems, and 
then specification errors. Four of these five were still important during 
the acceptance lest program with innate physical property problems 
becoming dominant. The qualification program appears to have been only 
marginally effective in screening these physical property problems, along 
with manufacturing process errors, circuit design errors, and specification 
errors (m order of importance to the acceptance program). 


TABLE 3-17. WORKMANSHIP PRIMARY FAILURE REPORT DEFECT TYPES 


Workmanship FR 
Defect Type 

Qualification (35.2 percent) 

Acceptance (51.4 percent) 

Quantity 

Percent of 
Total 

Quantity 

Percent of 
Total 

Test error 

1 

2.3 

15 

3.6 

Paper error 

5 

114 

19 

4.5 

Bad electrical connection 

1 

2.3 

66 

15.7 

Accidental handling 

1 

2.3 

0 

0 

Misuse 

1 

2.3 

1 

0.2 

Wrong part used 

0 

0 

16 

3.8 

Damaged wires/coax 

2 

4.5 

2 

0.5 

Contamination 

5 

11.4 

26 

6.2 

Bad alignment 

2 

4.6 

21 

5.0 

Installation/assembly 

26 

59.1 

255 

60.6 

Total 

44 

- 

421 

- 


Note: Usually, accidental handling and damaged wires/coax wa.e l&T induced, not 
"workmanship." 
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The primary and I&T acceptance FR data at both unit and systems 
levels for all programs was classified by type and subsystem to study the 
variances between subsystems. These data are presented in Table 3-20. 
Generally, 40 percent of all FRs were workmanship. 22. 5 percent part, 

13. 0 percent design, and significantly, 22. 5 percent were I&T induced. By 
subsystem, both power and propulsion had high numbers of l&T type FRs when 
compared to the other subsystems (36 and 47. 4 percent, respectively). The 
propulsion subsystem had a very low (5. 3 percent) proportion of part 
problems (no electronics), and the power subsystem had very few design 
(4 percent) problems. The telemetry and command (T&C) digital subsystems, 
and the attitude control subsystems, as expected because of a high con- 
centration of electronics, had the highest concentration of part problems -« 

33, 6 and 34. 2 percent, respectively. 


TABLE 318. PARTS PRIMARY FAILURE REPORT DEFECT TYPES 


Part F R 
Defect Type 

Qualification (12.8 Percent) 


Acceptance (29.1 Percent) 

Quantity 

Percent of 
Total 


Quantity 

Percent of 
Total 

Workmanship 

3 

18.8 


16 

6.7 

Contamination 

3 

18.8 


41 

17.2 

Mfg. process 

7 

43.8 


73 

30.7 

Design 

3 

18.8 


13 

5.5 

Unknown 

0 

0 


95 

39.9 

Total 

IS 



- 


238 

- 


TABLE 3-19. DESIGN PRIMARY FAILURE REPORT DEFECT TYPES 



Qualification (49.6 Percent) 


Acceptance (16 7 Percent) 

Design FR 
Defect Type 

Quantity 

Percent of 
Total 


Quantity 

Percent of 
Total 

Wrong part 

0 

0 


1 

0.7 

EMI 

0 

0 


1 

0.7 

Physical properties 

1 1 

17.7 


48 

35.0 

Specification error 

8 

12.9 


20 

14.6 

Process error 

11 

17.7 


31 

22.6 

Circuit design 

16 

25.8 


24 

17.5 

Mechanical installation 

16 

258 


16 

7.3 

Analytical error 

0 

0 


2 

1.6 

Total 

62 

. 

— 

1 

137 

- 
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TABLE 3-20. PRIMARY AND l&T ACCEPTANCE FAILURE REPORT DATA 
BY TYPE AND SUBSYSTEM 


I 

J 

1 



Failure Report Type 

Approximate 

Subsystem 

Unknown 

Workmanship 

Pan 

Design 

l&T 

Totals 

ol All Units -o/o 

Communications 

16(4.0) 

163140.3) 

97(24) 

45(11.1! 

83(20.5) 

404(38.2) 

49.4 

T&C RF 

2(1.5) 

50137.9) 

24(18.2) 

36(27.3) 

20(15.21 

132(12.5) 


110 

T&C digital 

211.3) 

61 140.1) 

51(33.6) 

11(7.2) 

27(17.8) 

152(144) 


(Combined) 

Attitude control 

2(1.8) 

39135.1) 

38(34.2) 

12(10.8) 

20(18.0) 

111(10.5) 

36 

Power 

0(0 

28137.3) 

17(22.7) 

3(4.0) 

27(36.0) 

7517.1) 

7.0 

Propulsion 

1(1.3) 

27135.5) 

415.3) 

8(10.51 

36147.4) 

7617.2) 

13.9 

Antenna 

0(0) 

17(43.6) 

3(7.7) 

13(33.3) 

6115.4) 

3913.7) 

3.6 

Spacecraft 

hartiets 

0(0) 

28(58.3) 

3(6.2) 

5(10.4) 

12125.0) 

4814.5) 


113 

Structural and 
mechanical 

0(0) 

6(42.9) 

0(0) 

3(21 4) 

5(36.7) 

14(1.3) 


(Combined) 

Ail others 

. 

0(0) 

2(33.3) 

1(16.71 

1(16.7) 

2(33.3) 

6(0.6) 

0.2 

Total 

23(2.2) 

421(39.8) 

238(22.5) 

137(13.01 

238(22.6) 

1057(1000) 

100 


NOTES: 1) Unit and systems level FRi from all programs 

21 ( ) « % of total 


The spacecraft harness subsystem was segregated from the structural 
subsystem because of the high number of FRs generated at the systems level. 
Of the 110 primary FRs of Program L at the systems level, 33 were wire 
harness (30 percent), and of 38 Ifc T FRs, 7(18 percent) were against the 
harness. The harness group was less significant on Programs Z and 3, 
where only two (6. 4 percent) primary and two I&-T (14 percent) FRs were 
reported. The total number of FRs per subsystem can be normalized, based 
upon number of units. As shown in Table 3-2, a weighted calculation 
indicates that approximately 50 percent of all hardware from which FRs 
could have been produced were from the communications subsystem. There- 
fore, 38. 2 percent of all programs primary and I&T FRs from the com- 
munications subsystem shows a better than average performance. Poor 
performers include the attitude control, 10. 5 versus 3. 6 percent; T&C 
(both) 27.7 versus 11 percent. Other "good" performers include propulsion, 
7, 2 versus 13, 9 percent, and structures, 5. 8 versus 11,3 percent. 


3. 5 ORBIT FR ANALYSIS 

All of the reported orbital problems have been coded and classified 
in a manner similar to that used for the primary FR data base at unit and 
systems levels. The data has been divided_intn_ha sic groups: 1) unique 

problems (those which occurred only once on a program or are the first 
occurrence of several of similar problems on the same and/or subsequent 
(time) spacecraft); and 2) duplicate problems (previously reported symptoms 
with the same determined cause but not necessarily on the same serial 
numbered spacecraft). A third group called operator error has been added 
to cover a single problem on Program 1 wherein a ground controller caused 
a major outage of payload communications through the use of improper 


1 

1 
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procedures. These groupings are summarized on Table 3-21 for the two 
programs (Programs 2 and 3 are combined). The percentage data shows 
the similarity between the programs. Approximately 50 percent of all 
problems are duplicates. Tnis would imply generic defects which cause 
repeated occurrences of the savne problem on several spacecraft within a 
program. The problems either did not occur or were not identified in time 
to enable the correction of subsequent spacecraft prior to launch. 

Of the unique problems, 52 percent were determined to be design 
defects. This implies that the test program was relatively ineffective in 
screening design defects. In fact, the vast majority of the design problems 
were not tested for during the acceptance test program. These problems 
involved life, static charge/discharge, orbital alignments, and zero gravity 
conditions, deficiencies for which the test program was not designed to detect. 
Therefore, the problem is one of overall test program design. It should also 
be noted that the data of Table 3-21 reflects almost 44 orbit years of exper- 
ience on the 12 spacecraft in orbit (see Table 3-1). 

The criticality of any residual problem is an important criteria in 
evaluating the effectiveness of a test program. This was evaluated as a 
function of this effect and is presented in Table 3-22. Anomalies, periodic/ 
intermittance, and hard failures occurred with the same frequency. All of 
the catastrophic problems (i. e. , loss of spacecraft or payload function of 
the spacecraft) occurred on Program 1 and were associated with a single 
type of unit. This was a life/design problem which occurred on a particular 
lot of units which were all launched prior to detection. Many of the orbit 
problems were not critical (36 percent), (e. g. , loss of a single telemetry 
channel). Others were operationally overcome by special ground station 
command sequencing (11 percent). The remaining problems (39 percent) 
were mission degrading in that some critical function (payload) was lost, 
impaired, or interrupted. 


i'ABLE 3 21. ORBIT FAILURE REPORTS FOR ALL PROGRAMS 


Type 

Program 

Total 

1 

— 
2 & 3 

Unique problems 


.. 


Unknown 

111.5)* 

1(4.5) 

2(2.2) 

Workmanship 

212.9) 

1(4.5) 

3(3.3) 

Part 

13(19.1) 

4(18.2) 

17118.9) 

Design 

18(26.5) 

5(22.71 

23(25.61 

Duplicates of above 

33(485) 

11(50.0) 

44(48.9) 

Operator error 

1(1.5) 

0 

1(1.1) 

Total 

68 

22 

90 


’Percentage shown in parentheses; the data have not been normal i/ocl 
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Not critical 

Operationally] 

overcome 

Mission 

degrading 

Catastrophic 

Total 



; r 

All same unit on Program , 

Operator error on Program 1 not included 




4. TEST EFFECTIVENESS 


4.1 INTRODUCTION 

Fur this study, test effectiveness has boon generally defined as tin* 
number of failure reports (FRs) generated in a given tost screen to the num- 
ber of FRs (i. e. , defects) which were available to be screened. The analy- 
sis was approached from two perspectives. First, "Specific Test 
Effectiveness” was calculated to assess the effectiveness of a particular test 
in detecting defects peculiar to that environment (o, g. , the effectiveness of 
unit vibration tests in screening the total FR group which was generated 
during unit, system, and launch vibration. Second, a "General Test Effec- 
tiveness ’ was calculated to assess the effectiveness of a given tesl - rclativc 
to the total FR population without regard to screening eligibility li, e. , all 
remaining FRs were considered available to the next test environment). 

Using both of these approaches, comparisons wore made between the individ- 
ual unit and systems level tests. A methodology is outlined to vary the 
content of tile total test program in response to the observed FR rates. 


4. d DATABASE 

The primary tits for lest and in-orbit from Programs 1 through 1 
w. e i e evaluated and classified in terms of the tests or particular environment 
where the !• Rs originated. Test data were taken from the summaries of 
section and in-orbit data were taken from section 1.4. For this analysis 
tin- qualif ic at ion and acceptance test data were regrouped in the following 
categories: 


Unit Tests 

Initial ambient 

Vib ration 

The rmal vac uum/ 
temperature cycling 

Final ambient 


Systems Level Tests 
DCTV/FSTV and initial 1ST 
Vibration and SPT 
Thermal vacuum 

Final 1ST 
I munch ope rat ions 


All unit tests 


TABLE 4-1. PROGRAM 1 IN-OR8IT PROBLEM SUMMARY 


! Problem 

Classification 

Number of 
Different Problems 

Number of 
Repetitive Problems 

Total 

Vibration 

1 

_ 

1 

Thermal vacuum 

20 

8 

28 

Not tested for 
during system 
test 

14 

25 

39 

Totals 

35 

33 

68 


The in-orbit data were similarly divided into three categories: 

1 ) vibration related (launch), 2) thermal vacuum, and 3) all problems 
(totals). The nature of the in-orbit data suggested further categorization. As 
shown in section 3. 5, many of the in-orbit problems were duplications of gen- 
eric deficiencies. In addition, a significant number of the in-orbit problems 
could not have been screened by the systems level acceptance test program as 
defined. A summary of the in-orbit problem data for Program 1 is presented 
in Table 4-1. The criticality of the in-orbit FRs is also of interest for analy- 
sis and, therefore, additional sorts were de-fined as follows: 1) catastrophic 

degradation in performance of the communications payload, 2) unique prob- 
lems, and 3) significant degradation of any spacecraft performance parameter. 
The total test effectiveness data base for the Specific Test Effectiveness 
evaluation is presented in Table 4-2. It includes, from all three programs, 
qualification and acceptance test and in-orbit data on vibration, thermal 
vacuum /thermal cycling, all tests, and the special categories of in-orbit 
problems defined above. 


4.3 SPECIFIC TEST EFFECTIVENESS 

The equations which define each "Specific Test Effectiveness" calcu- 
lation are summarized in Table 4-3. The results, using the data base of 
Table 4-2, are presented in Table 4-4, 

4.3.1 Qualification Tests 

The test effectiveness calculations for qualification tests are limited 
to the unit level because the total FR population is assumed to be those FRs 
actually generated in the combined unit and systems level test programs. With 
reference to Table 4-4, Program 1 unit qualification tests (all tests) wore 
significantly less effective than Programs 2 and 3 (3t> versus 72 percent). 

Tins is probably due to the increased complexity of Program 1 , which 
resulted in a greater number of relatively subtle design and interface prob- 
lems. The learning curve relative to test technique is also a likely signifi- 
cant factor, In any e\cnt, for a new complex spacecraft, tnc data suggests 
that systems level tests will probably be substantially more important than the 
unit level. From Table 4-2, the Program 1 systems level qualification test 
program screened more problems than unit level tests by a factor of 
(oO versus 33 FRs). For the combined programs, systems level tests are 
more important by a factor of 1.2, 
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TABLE 4-3. TEST EFFECTIVENESS EQUATIONS 


SPECIFIC TEST EFFECTIVENESS 1 


Equation 


Thermal vacuum/thermal cycling 


Total vibration FRs (unit or systems test) 

1 All subsequent vibration FRs 

Isyitems test arid in-orbit t Total Vibration FRs (unit or systems tett) 
or only in-orbit) 

Total therm al vacuu m/thtrma l cycling FRs (un it or systems taitl 

* Total thermal vacuum/thermal cycling 
thermal cycling (systems tett and * FRs (unit or systems tett) 


in-orbit or only in-orbitl 


AM FRs (unit or systems te tt) 

3 ' All subsequent FRs (systems test A ll FRs (unit or systems test) 
and in-orbit or only nvorbit) 


• Total communications 
loss 


All FRs (systems test) 

T£ * 

4 All subsequent in-orbit v u B 

FRs h 


‘ All FRs (systems test) 


e Unique problems 


• Performance degradation 
(all subsystems) 


Unit ambient 


Unit vibration and SPT 


Unit thermal vacuum/ 
temperature cycling 


All FRs (systems test) 

C 5 ' Number of operational problems + FRj , „ M , 

revealed 


All FRs (systems test) 

6 * Number ol failures vvhich degraded + A I1 FRs (systems test) 

performance or mission 

GENERAL TEST EFFECTIVENESS* 

Equation 


Total initial ambient unit FRs 

: 7 = All subsequent FRs (unit test, t Total initial ambient unit FRs 

systems test, and in-orbit) 


Total vibration end post vibration performance test FRs 
8 All subsequent FRs (unit test. Total unit vibration end post vibration 

systems test, and in orbit) performance test FRs 


Total unit thermal vacuum/thermal cycling FRs 

9 All subsequent FR* (unit test. Total unit thermal vacuum /thermo) 

systems test and in-orbit) cycling FRs 


♦The qualification tett effectiveness equation: do not include in-orbit failures 
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Table 4-3 (continued) 


T«tt 

Unit final ambient 


Equation 


Systems test initial ambient 
DCTV/ESTV and first 1ST 


Systems test vibration and SPT 


Systems test thermal vacuum 


Systems test final 1ST and Cape 


TE 


10 


Total final unit ambient FRs 


All subsequent FRs (systems test 
and in-orbit 


Total final unit ambiant FRs 


Total initial ambient, DCTV/ESTV and first 1ST FRs 

All subsequent FRs (systems , Total initial ambient and DCTV/ESTV 
test end in-orbit) and first 1ST FRs 


TE 


12 


Total systems test vibration and SPT FRs 


All subsequent FRs (sysiems 
test and in-orbit) 


Total systems test vibretion and SPT FRi 


TE 


13 


Total systems test thermal vacuum FRs 


All subsequent FRs (systems 
test and in-orbit) 


+ Total systems test thermal vacuum FRs 


TE 


14 


All systems test final 1ST and Capa FRs 


All subsequent 
in-orbit FRs 


+ Alt system? test final 1ST and Cape FRs 


Test 

Unit initial ambient 


Unit vibration and SPT 


Unit thermal vacuum/ 
temperature cycling 


Unit final ambient 


Systems initial ambient and 
DCTV/ESTV and 1ST (first) 


DESIGN TEST EFFECTIVENESS 

Equation 


TE 


IS 


Total initial ambient unit design problems 
All design problems in unit, systems test, and in-orbit 


TE 


16 


Total vibration and post vibration unit design problems 

All design problems in subsequent Total unit vibration and post vibration 
tests and in-orbit design problems 


Total unit thermal vacuum/thermal cycling design problems 
All design problems in subsequent Total unit thermal vacuum/thermal 

tests and in-orbit cycling design problems 


TE 


18 


Total final unit ambient design problems 

All design problems in subsequent + Total final unit ambient design 
tests and in-orbit problems 


Total initial arnbi. "'TV/ESTV and first 1ST design problems 

Design problems in subsequent Total initial ambient, DCTV/ESTV and first 
tests and in-orbit 1ST design problems 



Table 4-3 (continued) 


Test 

Systems test vibration 
and SPT 


Systems test thermal vacuum 


Systems test final 1ST and 
Cape 


Equation 


Total systems test vibration and SPT design problems 

All design problems in subsequent Total systems test vibration and 
tests and in-orbit SPT design problems 


Total systems test thermal vacuum design problems 

AH design problems in subsequent tests + Total systems test thermal vacuum 
and in-orbit design problems 


TE, 


22 


All systems test final 1ST and Cape design problems 

All design problems 


in-orbit 


+ All systems test final 1ST end Cape design problems 


TABLE 4-4. SPECIFIC TEST EFFECTIVENESS ESTIMATES (PERCENT) 



Unit Test Effectiveness 

System Test Effectiveness 

All Tests 


Program 

Program 


Teats 

1 

2 and 3 

1, 2 and 3 

t 

2 and 3 

1. 2 
and 3 

Programs 
1, 2. and 3 


Accept 

Qual* 

Accept 

Quel 4 

Accept 

Qual * 

Accept 

Accept 

Accept 

Accept 

Vibration 

se 

22 

77 

44 

93 

27 

67 

too 

83 

99 

Thermal vacuum/ 

temperature 

cycling 

69 

39 

76 

76 

70 

47 

30 

76 

38 

81 

All tests 

78 

36 

72 

72 

76 

46 

66 

55 

55 

90 

• Total communi- 
cations lots 

— 


_ 


_ 


99 

too 

99 


• Unique problems 

- 


- 


- 


71 

71 

71 


• Performance degra- 
dation (all 
subsystems) 











All data 

- 


- 


- 


68 

79 

70 


Items tested 
for 

- 


- 


- 


77 

— 

84 

79 



‘Based on tot jl FRs for the test program 


For qualification vibration tests, the systems level screen on Pro- 
gram 1 was more important by a factor of 3,6 (25 versus 7 FRs), and 2.7 for 
the combined programs, For the combined programs, a total of 33 percent 
of all qualification FRs (unit and systems leveL) were generated in a vibra- 
tion test environment. This relatively large fraction (vibration tests are 
only 12 percent of the total acceptance test FR group) is partly because many 
structural elements are tested for the first time at systems level. 
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For qualification thermal tests, the systems level screen on Pro- 
im P ort a. nt by a factor of 1. 6, and 1. 1 for the combined 
” ' For the comb ined programs, a total of 14 percent of all qualifi- 
cation FRs were generated m a thermal test environment (thermal tests are 

19 percent ol the total acceptance test FR group). 

li Th f e jf ecti vene SS of the qualification test screen relative to lesign 
problems is discussed in Section 4. 5. 8 

4.3.2 Acceptance Tes ts 

The Specific Test Effectiveness calculations for acceptance tests are 
ba.ed on the entire FR history for .he flight spacecraft, taclSdtag In-orii* 

from tSs n da'te h dite t T F "“ Spl * CeCraf ‘ tor b°th programs was excluded 
from this data base because no m-orbit data exists (the booster failed for F-6 

la P unc e hed u ‘ ^ *?! £t of Program 3 has not been 

rnn.l.f* Th eff ® ctlveness o£ th e acceptance test screens was relatively 
consistent across the programs with a combined program test effectivenesses 

tL test ^°pro U g"am. StS ’ ^ £ ° r SyStemS teSt5 ’ and 29 P^"' = *»r 

The vibration acceptance test screen was highly effective at 93 nercent 

tlZZrlZ n r rce "‘ £ f r S >' S *' m ' '“*»• and for , hi 

A total of ^ i? Y one vibration problem occurred for 12 spacecraft Launches. 

The dir a ^ PerC i acceptance test FRs occurred in a vibration 

“ . The data strongly suggests that a three-axis vibration test at spacecraft 

nc F l Pr reqU ^ ed ; F ° r 311 systems level vibration test 3 P (includ- 

.® . P acecra ft)i a total of only six FRs were generated, with five occur- 

. \ n tbe P^st- vibration performance test (SPT) and one occurring during the 
initial vibration test which is along the Z-axis (see Table 3-12) The risk 

UVel aCCeptanCe Vibrati ° n ^ 

unit lev o^y^^perc^nt^ff^ctive^ at^y st^ms^fever^T^he^co^ibin^d Jest 
programs were 81 percent effective. With reference to the Program 1 in-orbit 
FR summai y in Table 4-1 , note that 28 (41 percent) In-orbit problems 
occurred which could have been detected in either unit or systems level 
thermal test !" 8 These problems included telemetry instrumentation errors 
circuit desLgn deficiencies, noisy earth sensor bolometers, and RF anomalies 
However, a total of 39 (57 percent) of the in-orbit problem; could not hiSi 
been screened by the test program because of test technique deficiencies. 

These included static discharge problems, early wearout of life limited items 
thermal deformation of structures, ground station operator errors and zero-’ 
ffthT y ., ef£ect f’ Generally, the deficiencies in test technique resulted from 
either the omission of tests such as static discharge or TWT life tests or 

low'effecVv 5t Umts V? this entipe operating mode. Therefore, the relatively 
low effectiveness of the systems level thermal vacuum environment is n ,b 
ably a result of test technique deficiencies ; nd not total test time in that 
environment. The limited effectiveness of increased test time in the thermal 
vacuum environment is presented in more detail in section 6 of this report 
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99 ‘yjcVntfM »«.l. 

the 4 cc.pt»i,“ t..?^ "' ”'"‘>*"i»m * hich could ™,T J a " d waa «>« 

no fa,lu ;; whirt 3 b rvr/s„ b / e , 

‘ - FRs with 
>=-«*d aT e excluded’ ST“* * U FRe - » •>>« PR. “hid ' ^ " mbi "' d pro- 

tor performance rjltS™ ^-‘iveness^crTaec S°* 


“• 4 GENERAL test effectiveness 


* 1*JWU 

if t Ta n bi a e r 4 a 5 S ° SUm ^ a ^ed fn Te , st Effectiveness- caicu 

-eiltiveett %*?“'** in TabIes 4-6 and 4 Tt*’ ^ the 
- " bi ‘- ‘ ne5s 04 u "“ - 

1“a r» i i- > 


^LE4-e. GENERAL ffSTgFFECTlVfWESS DATA BASE 


Unit 

Ambient (initial)* 4 * 

Vibration and SPT 

Thermai vacuum/temperature cycling 
I Ambient (final) 

Systems 

SfJ^ocnwnr.^ 

Vibration and SPT 
Thermal vacuum 

1ST (final) and Cape (acceptance only) 
J^ bit problems - 10 year project ion 



2*?^ spacecraft, 

same period. 1 e pred,cted f «<lure rates for spacecraft W hth MS ' d4 , rm9 the relative 

Jhe shelf i 5 excluded spacecraft wh, ch We , a launched in the 


. .same period. 

. . . J he she H is excluded " c 


4-8 


i 


I 

I 

I 

1 
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i 

1 

*0* 

1 

1 

I 
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TABLE 4-6. GENERAL TEST EFFECTIVENESS OF QUALIFICATION AND ACCEPTANCE TESTS 


Tests 

Test Effectiveness, Percent 

Qualification 

Tests 

Acceptance Tests' 

Program 

Program 

1 

2 

1 

2 and 3 

Unit 





Ambient (initial) 

21 

44 

43 

43 

Vibration 

11 

22 

22 

7 

Thermal vacuum/cycling temperature 

8 

21 

32 

25 

Ambient (final! 

2 

18 

5 

2 

Combined unit tests 

35 

72 

71 

61 

Systems 





Ambient, DCTV/ESTV, and 





1ST (initial) 

- 

- 

32 

18 

Vibration and SPT 

- 

- 

2 

4 

Thermal vacuum 

- 

- 

7 

3 

1ST (final) and Cape (acceptance onlyl 

- 

- 

27 

10 

Combined systems tests 

- 

- 

55 

31 

Combined unit and systems tests 


- 

87 

73 


'General unit ambient acceptance test effectiveness was calculated with the following equation: 

TE ? - 304/703 - 0.43 (43 percent). All other acceptance test effectivenesses were calculated in the 
same way. 


TABLE 4-7. SUMMARY OF ACCEPTANCE TEST SCREENING EFFECTIVENESS 
FOR COMBINED PROGRAMS 


Tests 

Number of 
FRs 

Remaining 

Defect 

Population 

Test 

Effectiveness, 

Percent 

Percent of 
Test FRs 

Percent of 
Systems 
Test FRs 

Unit 

Initial ambient 

406 

942 

43 

51.6 


Vibration 

99 

536 

18.5 

12.6 

— 

Thermal 

131 

437 

29.9 

16.6 

_ 

Final ambient 

11 

306 

3.6 

1.4 

- 

Unit Total 

647 

942 

68.7 

82.2 

I 

Systems 

DCTV 

It 

295 

3.7 

1.4 

7 9 

Initial 1ST 

7) 

284 

25 

9 0 

50.7 

Vibration and SPT 

6 

213 

2.8 

0.8 

4.3 

Thermal vacuum 

11 

207 

5.3 

1.4 

7 9 

Final 1ST and Cape 

41 

196 

20.9 

6 2 

29 3 

Systems total 

140 

295 

47.5 

17.8 

100 

All tests total 

"*4 

CO 

942 

83.5 

too 

- 

Orbit (1 year estimate) 

112 

155 

72.3 

— 

... 

Orbit total (10 years) 

155 

155 

100 

- 

— 

Grand total 






(14 spacecraft) 

942 

- 

- 

- 

- 
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PERCENT 



UNIT UNIT UNIT UNIT SYSTEMS SYSTEMS SYSTEMS SYSTEMS SYSTEMS IN ORBIT 

INITIAL VIBRATION THERMAL FINAL OCTV/ESTV INITIAL VIBRATION TV FINALIST ( 1 YE ARI 

AMBIENT AMBIENT 1ST ANOSPT AND CAPE 


FIGURE 4-1. ACCEPTANCE TEST SCREENING EFFECTIVENESS FOR COMBINED PROGRAMS 





4.4.1 Qualification Tests 


The test effectiveness calculations for qualification tests are limited 
to the unit level because the total FR population is assumed to be those FRs 
actually generated in the combined unit and systems level test program. 

With reference to Table 4-6, the Program 2 combined unit qualification tests 
were significantly more effective than Program 1 (72 versus 35 percent). 

This probably is a result of tne higher complexity of Program 1 which 
resulted in a greater number of subtle design and interface problems com- 
bined with the learning curve relative to test technique. The Program 2 sys- 
tems level prototype spacecraft tests benefitted from the Program 1 test 
experience. However, for a new complex spacecraft, the data suggests that 
systems level prototype spacecraft tests will probabLy be substantially more 
important than the unit tests by almost a factor of two (60 versus 32 FRs). 

The unit ambient test screen was the single most effective unit test 
in both programs (21 percent for Program 1 and 44 percent for Program 2). 
The unit qualification vibration test was the second most effective unit test 
and was 11 percent effective in Program 1 and 22 percent effective in Pro- 
gram 2. The thermal vacuum test was more effective for Program 2 (21 per- 
cent) than for Program 1 (8 percent). The final ambient test was only 
2 percent effective in Program 1 , whereas it was 18 percent effective in 
Program 2. 


4.4.2 Acceptance Tests 

The general test effectiveness calculations for acceptance tests are 
based on the entire FR history for the flight spacecraft, including the pre- 
dicted in-orbit performance for 10 years. The ground test FRs include the 
F-6 spacecraft data. A predicted 10 year failure estimate has been included 
in the in-orbit failure estimate to correct for the loss of the F-6 spacecraft. 
Table 4-5 describes this calculation procedure. The Programs 2 and 3 in- 
orbit total failure estimate has been similarly corrected to include a predic- 
tion of the yet unlaunched F-6 spacecraft in-orbit failures. The effectiveness 
of the acceptance test screens was relatively consistent across the programs 
with a test effectiveness for units of 71 percent in Program 1 and 61 percent 
in Programs 2 and 3; 55 and 31 percent, respectively, for systems level 

tests, and 87 percent for Program 1 and 73 percent for Programs 2 and 3 

for the entire test program. 

The Programs 1 through 3 unit initial ambient tests were the most 
effective tests (43 percent) performed at the unit level (see Figure 4-1). The 
unit thermal tests were the second most effective tests with 29.9 percent 
(32 percent Program 1 and 25 percent Programs 2 and 3). The data indicate 
that these tests provide the best opportunity for defect detection and 
co erection. 

The systems level ambient, DCTV/ESTV, and initial 1ST provide the 
next most effective defect detection test, The test was 32 percent effective 
on Program 1 and L8 percent effective on Programs 2 and 3, with 28 percent 
overall. 
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The systems level final 1ST and cape tests were 20.9 percent effective 
Z pe . rcent on Program 1 and 10 percent on Programs 2 and 3) The unit 
vibration test was 22 percent effective in Program 1 and onfy 7 pelcL^kc- 
ve m Programs 2 and 3. The combined test effectiveness of the unit final 
ambient, systems Level vibration, and systems thermal vacuum tests 
ranged between 2. 8 to 5. 3 percent. The data indicate that these tests are not 
very effective in sc reening defects and failures. They suggest that reduced 
vibration and thermal vacuum testing at the spacecraft acceptance level Zould 
be implemented without a significant effect on the fraction defective in a 
delivered spacecraft, c a 


4. 5 TEST EFFECTIVENESS FOR DESIGN PROBLEMS 

Th i C e< * uations . whi ch define each "Design Test Effectiveness" calcula- 

rlhlVI 8 Sun ? marize l^ Tab *e 4-3. The results, using the data base of 
Table 4-8, are shown in Table 4-9. 

4. 5. 1 Qualification Tests 

. , . T ^ e t ** t effectiveness calculations for the ability of the qualification 

tests to identify design deficiencies are based on the FR history^or all 

* P r C o a t (pr0t0type a " d £lisht)> and inc,udes in-orbit performance-ia 
, e * C ^ ll > Ldesi g n problems have been included. The design test f 

jn detecting 

systems devel vibration test had ui„u \ ? e ", encctive. 1 he Program 1 

Ws&gmM mmm? 
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TABLE 4 8. PROGRAMS 1 THROUGH 3 DESIGN PROBLEMS SUMMARY BY TEST PHASE 


Spacecraft 


Unit Test | 

Initial 

Ambient** 

Vibration 

TV/TC 

Final 

Ambient 


Systems Test 


Initial 
Ambient 
1ST and 
DCTV/ESTV 


Vibration 
and SPT 


Orbit 

Final Launch Unique 
TV 1 ST Operations Problems 


Oual 

Ft 

F-2 

F-3 

F-4 

F S 

F-6 

F? 

F-8 

Unassigned 


Subtotal 


F-6 

Shelf 

Unassigned 


Subtotals 


The 9 FRs written against a single capacitor have been considered a single problem. 

The unit FRs where the test phase unknown were included in the initial ambient unit test. 


4.5.2 Acceptance Testa 

Because of the relatively high numbers of design Fils on the early 
spacecraft, the acceptance design test effectiveness calculations were based 
on the ability of the first three flight spacecraft of Programs 1 and 2 to 
identify those design deficiencies which were undetected by the qualification 
programs. The combined unit and systems lcvc' test were 49 percent effec- 
tive. Therefore, approximately 75 percent of all design deficiencies were 
found after the testing of the first four spacecraft (prototype and three flight) 





















































„ 8LE49 GENERAL TEST rnCTVBB. FOR PROBLEMS 


Design Test eff ectiveness. Percent 
"Qualification I Acceptance 


Program 1 Program 2 


Prograr s 1 and 2. 
F-1, F-2, and F-3 
Spacecraft 


Unit 

Ambient 

Vibration 

Thermal vacuum/temperature cycling 
Ambient 

Combined unit tes t 
, Systems 

j Ambient, DCTV/ESTV, and 1ST (first) 
Vibration and SPT 
Thermal vacuum 
1ST (final I 

Combined system s test 
Combined unit and systems test 


The unit t..t. were pe rcert>. The si "f c ™* u p.r. 

th-an the systems lcve, ^ (22 percent). Unit thermal tests Mitral 

tive test was initial- amb en . 

cent effective and the Y The rema tmng tests we V 

1ST were 10 percent effective. 

(1 to 4 percent). n rl svsterns level acceptance 

The first three n^VHSS.™ <lcsiB ” P ''"' i’ena 0 ' " 

tes ts were surprisingly mPr levc ( qualification programs. ‘ tho iim , uin t 

the individual unit and 3 Y^c 1 • detecting design pn.btcms ^ 1 . 

to suggest that an important fact ^ Unt in detecting variety »f 

„( time spcnl »t »»«• -^? C conct.i.i<.n may 1» th»l 

u.» i« 

i. .»* ** « « - 

continue the testing. 
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4.6 RELIABILITY PREDICTIONS USING A TEST EFFECTIVENESS MODEL 

independent of the d.-<Wt " ‘ ’ 1 , * est screen effectiveness is 

are eligible for the screen). TheTest^’ &nd &U members ° f tile population 
effectiveness. The TE model fEauati 'll* 6 n ° l re< l u ^ re ^ to have equal 

environments. independent of the sequence of 

Let: 


then: 


T 

f 

n 

F 


N 


E # 
n 


total number of defects in the initial population 

number of defects screened in environment "n" 

total detect, screened by the combination of environments 

test effectiveness of environment "n M 


T- "C 1 f 
n=l 1 


f = E 
n n 


n - 1 


n=l 


n 


( 1 ) 


screened 1 ",! Lw n^H his . thc P^P^y that the number of detects 

linearly to th^omb?^”"™ io 

Let: 


E 

E 

T 

f l 

fv 


l 


effectiveness of test group r 
effectiveness of test group l 
total initial defect population 
defects screened in test group 1 
defects screened in test group Z 


Teat's Iltd’spacf.c Polmcdinri 4 ' f"o“ h ‘, P b V' w<,,-n Performance in Systems 
1975, 1,171 j 'ocd.,,^ of the Ins t, tut,. of Environ,,, s ’ lvm 
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then 


EjT 


f 


2 


= E 2 





Therefore, a convenient check of the applicability of the TE model for 
prediction purposes is to evaluate the relationship between the in-orbit fail- 
ure group for a fixed period (1 year wa3 chosen) and the systems level test 
group or the combined systems, and unit test groups, A linear relationship 
would be expected. Similarly, a linear relationship should exist between the 
systems level test failure group and the unit test failure group. 

These data are presented in Figure 4-2. Data Set "A" illustrates the 
relationship between in-orbit FRs and the combined tests FR group, Simi- 
larly, ”B" and "C" compare systems level test FRs with in-orbit and unit 
level, respectively. The FR data is summarized in Table 4-10, The linear 
relationships predicted by the TE model are apparent. The unit data indi- 
cates a bias of approximately 20 FRs which apparently have no reliability 
significance. The F4 and F8 spacecraft appear unusual but may only be 
indicative of the statistical variance. The defect analysis data presented 
earlier in this report (see Figure 3-4) also indicated F4 and F8 were atypical. 
A more detailed analysis of the data base is required, 

As mentioned earlier in this report, the FI spacecraft was retro- 
fitted to correct all known design problems and was exposed to a complete 
retest it systems level prior to launch. Given the test history of F*1 
(105 FRs; see Table 4-10) and the estimated effectiveness of the Program L 
combined test environments (0.87, see Table 4-6), a total of 8.6 failures 
would have been predirted_fax_the retest program. In fact, a total of 8 did 
occur. This data is shown as F1 r in Figure 4-2, The in-orbit (1 year) 
screening effectiveness is greater than systems level test (0.72 versus 0. 551 
and, therefore, 11 failures would have been expected if FI had been launched 
without the retest program. This is in good agreement with Figure 4-2, 

In summary, the data indicates that a-TE model could be used as the 
basis for reliability predictions and, therefore, could be used to adjust the 
content of a test program as required to achieve an acceptable residual 
defect group. If the number of test failures were too high for a particular 
spacecraft, the test program would be continued until acceptable FR rates are 
achieved.- The F-l retest program is a case in point. Consistent failure 
reporting criteria would be required and statistical limits for the TE mode! 
would have to be developed. The results of this study indic ate (hat such an 
approach is feasible. 
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TEST FAILURE REPORTS (UNIT OR SYSTEMS OR COMBINED) 


’ M * “ TH£flE W6R6 N0 FAILUHES IN ™ E F1RST OF ORBITAL OPERATIONS ON THE F-1 RETESTED SPACECRAFT 

•■THE FI WAS PREDICTED TO HAVE 11 IN-ORBIT FAILURES DURING THE FIRST YEAR IN ORBIT WITHOUT RETEST 
•’•THE F )r ACTUALLY HAD 8 FAILURES DURING RETEST 


FIGURE 4-2. PR0GRAM4-EAILURE RATE GROUPS COMPARISON 


TABLE 4 10. SUMMARY OF PRIMARY FAILURE REPORTS FOR PROGRAM 1 

I Unassigned FRs Excluded) 


Program 1 
Spacecraft 

Total 
Unit FRs 

Total 

Systems FRs 

Combined Total 
Test FRs 

Systems Level 
Retest FRs 

In-orbit* 
FRs (1 year) 

FI 

77 

28 

105 

8 

0 

F-2 

74 

18 

92 

_ 

14 

F-3 

52 

15 

67 

_ 

8.8 

F-4 

83 

6 

89 

— 

8.8 

F-5 

52 

10 

62 

— 

8.2 

F-6 

41 

15 

56 


_ 

F-7 

36 

8 

44 


5.3 

F-8 

57 

10 

67 


2 

Totals 

472 

110 

582 

8 

47.1 

Average 

per 

spacecraft 

59 

13.76 

72.8 

1 

6.7 ‘ * 


The number of in-orbit FRs for the first year in orbit was estimated from the orbital Duane plots 
_ ln Figures 5-2B through 5 98 or 5 16 for the F-8 spacecraft. 

‘The average was based on seven in-orbit spacecraft since the F-6 booster failed. 
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5. FAILURE RATES AS A FUNCTION OF TIME 


time dep T e h n e d:^^dr;l,d M 4l«;»1L , f ,J r da *f. w r 

the Duane reliability growth model was evawJd!'"''^'' ' ** applicabilit y 

have been a s s o c i a*t e d^itiTthe! r ^ e spt c ti'v e'" Ume failure sports (FRs) 
required data base. The test tlme-to^faifull ® * occurrence to establish the 
FRs and was estimated from either *>, ■ . , as used f° r the systems level 

test reports. Th^time l individual spacecraft test logs or 

since spacecraft "off" time during test^nnV^ 1 ? 0 " of the time-to- failure, 
etc. . was not extracted. The in orbit tim ” lgura ^ on * thll * d shift, weekends, 
customers at the time the event burred ? bTJV ^ the 

is current to the date of this report (March 1 ^1^7^' *' °n' in_orblt time 
times varied from a low of 475 2 test hours to a high of 1? 288* BY ?* ma test 
mulated m-orbit time varies between 15 593 and 53 Tvn t’ 288, and L accu “ 
spacecraft of the three programs it is diffL,,!* f 53, 173 . hours on the different 
fore, analysis was basically , n ? ‘ ‘° combine all data. There - 

accumulated time for systems level test ^ su "‘™ary of 

and combined is provided in Table 5-1. The total database (3 ' U77) ' 

12 »•“* l t~t experience and^lmo^ 44* tn-orbu' yea”, 5 . 

different graphs were' prf ^r^or^ac^silcei 8 ' fl thr °' 7 mo, ' e FRs >' lhr “ 
first, graph A. was a C niJ r!. spacecraft as applicable. The 

the second, was a Duane plot^f the in^orb^MDort^d 1 1 ** 1 daU ' Craph B ’ 
date as the initialization point. The third eraoh r * usln 8. eacb launch 
test and in-orbit plot, usine the t ' ? apb ls a com bined systems 

point and assuming that the in-orbit- Ha°f systems t . est as the initialization 

test (launch), it was not analyzed rigorously Sth^fhe D ^ 'Tl ° f s y stcms 
presented in the same format h°r°usiy with the Duane model, but was 

the cumulative tme'oTeacl e^ent^S re'^ott^d'’ 0 ? lilure '' ate versus log 10 Q f 
was used to fit least squares estimates forth*’* A wei 8 htod hnc a r regression 
The weighting scheme used for th r be a PP ro P riate Duane equations, 

failure of ealh apa T-raft a weight |* f Bivc the fir,. 

1 a Weiffht « 1. the second failure a weight of 2, the 
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Spacecraft 

Program 1 
Prototype 
F-1 
F-2 
F3 
F-4 
F-5 

F-e 

F-7 

F-8 

Program 2 
Prototype 
F-1 
F-2 
F-3 

Program 3 
F-4 
F-5 
F-6 


TABLE 5-1. SYSTEMS TEST. IN-ORBIT, AND COMBINED 
ACCUMULATED TIMES (HOURS) 


Systems Level 
Test (Launch) 


7,248 (EOT)* 
12,288 
5,184 
5,592 
4,752 
6.552 
6,700 
5,832 
7.848 

-7,536 (EOT) 
6,384 
6,864 
8,808 

5,832 

5.328 

5,038 (EOT) 

107,836 
(12 3 years) 


All times in hours; EOT — End of test 

nth failure a weight of n, e 
All data is assumed to fit t 


In-Orbit 
to 3-1-77 

Combined 
(If Applicable) 

N/A 

N/A 

16,693 

27.881 

53,173 

58,357 

45,552 

51,144 

44,764 

49,516 

41,347 

47,899 

N/A 

N/A 

30,835 

36,667 

19,973 

27,821 

N/A 

N/A 

37,756 

44,140 

33,813 

40,677 

15,943 

24,751 

25,228 

31,060 

20,936 

26,264 

N/A 

N/A 

384,913 

466,177 

(43.9 years) 

(53.2 years) 


n, etc, ; thus , 
fit the Duane 


giving the last event 
analysis model of the 


the most weight, 
form: 


- KT' 


where: 


cumulative failure rate 

total time in hours 

failure events during time T 

constant determined by circumstances 

growth rate (slope) 



When plotted on on log.. - log . paper, the model appears as a 
straight line of the form: 



= slope 


i 


Therefore, equation (1) reduces to: 

log 10 X = log 10 K ‘ Q ' log 10 T {Z) 

and regression fitting will determine the constant K and slope a. 

The failure rate at time t. isX..; this equates to instantaneous failure 
rate for any event as: 


F. dt 

-V- ■ KT 


where: 


= instantaneous fails per unit time at time t, 


Therefore: 


= K (!-«) t/ 


Using the above expressions, the Duane model can be interpreted for three 
different conditions: 


45 ’ 

- 4 — 2 


For condition 1, <*= +1; therefore, F. = 0 and X. = K/T. This is a 
limiting condition and a<+ 1 , 1 1 

For condition Z, a - 0; therefore, F. = K and X. = K (i. e. , the failure 
rate is a constant, not influenced by time. ). 1 1 

For condition 3, a = " 3 ; therefore, F { = 2KT and X. = KT. For this 
case, the failure rate linearly increases with time. This implies either an 
increasing test screen effectiveness or that the test article is degenerating 
as a result of test (i. e. , the defect population is increasing with time}. 

An understanding of these boundary conditions will assist in the eval- 
uation of the Duane plot's. 

Figures 5-1 through 5-15 contain the scatter diagrams for each 
spacecraft on Programs 1 through 3 where sufficient data existed. Figures 
A and B contain the weighted least squares Duane estimate curves. Fig- 
ure C contains only visual estimations of the curve (or curves) to fit the 
data, using straight lines. For this combined data no meaningful weighted 
least squares Duane estimate could be found for most spacecraft because 
of the marked contrasts in slope between early systems test and in-orbit 
data. A time reference from Table 5-1 has been provided on each graph. 

The operational time-to-date for B figures has been indicated but was not used 
as a data point for purposes of determining Duane curves; only data points 
corresponding to the occurrence of failure events were utilized in the Duane 
analysis . 

Table 5-2 contains the parameter estimates derived for each Duane 
analysis shown in the figures. This table also contains the number of data 
points utilized to derive each Duane curve estimate. It should be noted that 
where two or more failure events occurred at the same point in time, the 
total cumulative failure rate was calculated as one data point. 

A review of the spacecraft Duane curves and Table 5-2 reveals that 
systems level failure rates vary widely as a function of time. This is best 
seen in Figures 5-l8a and 5- 19a, wherein all systems level Duane curves 
have been redrawn on one figure for Program 1 and Programs 2 and 3, 
respectively. The slopes shown appear to represent all three Duane curve 
bounding conditions described above. Early spacecraft trend toward nega- 
tive slopes (-o) and later spacecraft approach the a - +1 limit. The final 
failure rate of systems level test (Xf st ) does tend to converge for all flight 
spacecraft (especially on Program 1) and gets lower with each succeeding 
spacecraft (fewer FRs per spacecraft, therefore, lower failure rate). 

When all of the in-orbit Duane curves are compared and plotted on 
Figures 5-18b and 5-19b, it is shown that all slopes are similar (positive ») 
and form a family of curves, again with the property chat successive space- 
craft generally have lower failure rates at any given time in orbit. There 
are three special cases for the in-orbit data. Both Program 1 F- 1 space- 
craft and Program 2 F-3 spacecraft have had no orbital anomaly reports in 
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over 15,500 hours of operation, and, therefore, have rot been shown. The 
F-5 spacecraft of Program 3 has not had an orbit problem in its last 11,000 
hours of operation. The F-5 spacecraft also has the Duane curve with a 
negative slope (-a). If one assumes a failure occurring on 1 March 1977 for 
this spacecraft, a worst case estimation line could be drawn through the cal- 
culated failure rate at 20, 936 hours. As a failure has not in reality occurred, 
the actual slope should be even more positive than that drawn on Figure 5 -15b. 
This estimator would appear to depict the F-5 spacecraft in-orbit performance 
more realistically than the Duane curve and was used in Figure 5 -19b. 


TABLE 6-2. SYSTEMS TEST AND IN-0R81T DUANE CURVE 
PARAMETER ESTIMATES 





Systems Test 



In-Orbit 


Spacecraft 

No. of 
Data Points 

Estimate 

for 

K-Constant 

Estimate 

for 

dr-slope 

No. of 
Data Points 

Estimate 

for 

K-Constant 

Estimate 

for 

»-slope 

Program 1 
Prototype 

33 

5X1 O' 5 

-0.61 

N/A 

N/A 

N/A 

F-1 


23 

0.512 

0.58 

None 

Insufficient data 

F-2 


1 1 

ixitr 10 

-2.03 

12 

3.09 

0.83 

F-3 


15 

0.0275 

0.23 

9 

1.74 

0.82 

F-4 


6 

0.00195 

-0.06 

8 

2.29 

0.85 

F-5 


9 

0.00041 

-0.16 

12 

0.955 

0.76 

F-6 


13 

0.457 

0.61 

N/A 

N/A 

N/A 

F-7 


7 

0.155 

0.55 

8 

0.309 

0.69 

F-8 


10 

0.145 

0.54 

3 

Insufficient data 

1 -Composite 

113 

0.126 

0.19 

45 

6.16 

0.78 

p rogram 2 
Prototype 

8 

0.00141 

0.02 

N/A 

N/A 

N/A 

F-1 


11 

0.0562 

0.37 

5 

0.00093 

0.16 

F-2 


2 

Insufficient data 

6 

0.0933 

0.58 

F-3 


6 

0.089 

0.53 

None 

Insufficient data 

Program 3 
F-4 

3 

9.5X1 0' 45 

-11.36 

3 

Insufficient data 

F-5 


3 

0.209 

0.67 

6 

0.000323 

-0.06 

F-6 


2 

Insufficient data 

N/A 

N/A 

N/A 

2-3 C 

amposite 

32 

0.0537 

0.26 

18 

0.0115 

0.26 


1 ) All equations are assumed to be of the form: X = KT~°! 

2} Number of data points are not equal to the number of events as some events occurred on the 
same day. 
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TM. HOURS” 

« SYSTEMS TESI ANO IN-ORBIT COMBINED 
FIGURE 5-15. PROGRAMS FS OATA 







The Da?ne curve provides a reasonable fit of the systems level test 
lailure rates. When this is combined with the visually estimated in-orbit 
continuous curve after launch, a reasonable approximation of each spacecraft 
total profile is obtained. These profiles have been drawn for every space- 
craft on Figures 5-16 and 5-17 for Program 1 and Programs 2 and 3, 
respectively. A convergence of the failure rates at the end of systems test 
(and a high correlation) with the in-orbit initial failure rate can be seen 
The two spacecraft which have had no failures in orbit were estimated on the 
individual C figures by the method of assuming a failure on 1 March 1977 
With the additional requirement that c<+l. The use of X f «. t as a predictor 'of 
initial nv orbit performance and the fact that the in-orbit slopes within each 
program are similar (0.78 for Program 1 and 0.26 for Programs 2 and 3) 
form a basis for future prediction of in-orbit performance. 

A composite of figures C was prepared as shown in Figures 5-20 and 
5-2 1 for Program 1 and Programs 2 and 3, respectively. These differ from 
■figures 3-16 in that only the visually estimated curves are shown, 

The varying slopes during the systems test phase are primarily caused 
by changes in the screening effectiveness of the various test phases and by 
the total number of defects presented to the systems level screen. This is 
an inherent weakness of the Duane model. However, as the spacecraft 
esign matures, the unit level and initial systems level tests become more 
effective and the cumulative failure rates assume a more classic Duane 
profile. 


v k® definition of a physical model which could provide a basis for the 

observed Duane trends is considered in Appendix B. A relatively simple 
model is shown to fit most of the empirical data evaluated in this study. 
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t END 0F IN-0R8IT ACCUMULATED 1 TIME — 

ESTIMATED (3-1-77) ~j:" 
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NOTES: 

1) ALL CURVES TO THE LETT Of LAUNCH ARE 
SYSTEMS TEST DUANE PLOTS 

2 ) ALL CURVES TO THE RIGHT OF LAUNCH ARE THE 
IN-ORRIT ESTi AATED CURVES FROM THE 
COMBINED DATA OF GRAPH "C" 
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FIGURE 5-16. PROGRAM 1 - COMBINED SYSTEMS TEST DUANE PLOTS AND IN-ORBIT ESTIMATES 
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FIGURE 5-20. VISUAL ESTIMA 
SPACECRAFT-SYSTEMS TEST 
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SCREENING EFFECTIVENESS OF SYSTEMS 
THERMAL. VACUUM TESTS 


The specific test effectiveness and general test effectiveness of 
thermal environments were discucsed in Sections 4. 3 and 4.4. The test 
effectiveness of systems thermal vacuum acceptance tests was very low at 
7 percent for Program 1 and 3 percent for Programs 2 and 3. A summary of 
systems level FRs is presented in Table 6-1. The test time -to-failure was 
determined for each of these systems level thermal vacuum FRs, and a fail- 
ure histogram is presented in Figure 6-1. The screening effectiveness of the 
environment decreases rapidly with time with most of the failures occurring 
in the first 60 hours of test (87 percent). 

The relative importance of this acceptance test screen is illustrated 
in Figure 6-2 for Program 1 systems thermal vacuum with DCTV excluded. 

The test effectiveness of the thermal vacuum environment (7 percent) is a 
factor of four less important than the following final ambient tests (27 percent) 
and a factor of ten less important than the first year in-orbit. The relatively 
low test effectiveness of the thermal vacuum environment may be explained 
ir. terms of the test not inducing sufficient stresses to cause failure, or there 
being enough novelty, e.g. , changes in environment and operating modes to 
cause marginal hardware tea fail. A cumulative failure rate curve was pre- 
pared, Figure 6-3, for the thermal vacuum FR data. From Figure 6-3 it is 
shown that additional time in thermal vacuum environment is of little value 
because the instantaneous failure rate ( \ i > is nearly zero ( o' - + 1 ) after 70 hours. 
This is equivalent to a test effectiveness asymptotically approaching 0. 076. 

As shown in Figure 6-2, the cumulative test effectiveness for Program 1 
reaches 0. 06 after only 60 hours in test. The data strongly suggests that the 
test technique should be improved to add novelty and significantly increase 
the test effectiveness relative to the first year in-orbit. Time in the 
environment is not necessarily the most important factor. 


TABLE 6-1 . THERMAL VACUUM FR SUMMARY 
(See Tables 3-1 1 and 3-12) 


Thermal 

Vacuum 

Test 

Program 1 

Programs 2 ard 3 

Totals 


Dual 

Accept 

Qual 

Accept 


OCTV/ESTV 

N/A 

5 

1 

6 

12 

Systems TV 

7 

9 

0 

2 

18 

Totals 

7 

14 

1 

8 

30 
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FIGURE 6-1. PROGRAMS 1 THROUGH 3 - DCTV/ESTV AND SYSTEMS 
LEVEL THERMAL VACUUM FAILURE REPORTS VERSUS TIME 



FIGURE 6-2. PROGRAM 1 TEST EFFECTIVENESS COMPARISONS 



FIGURE 6-3. CUMULATIVE FAILURE RATE FOR THERMAL VACUUM TESTS 
AT SYSTEMS LEVEL (DCTV/ESTV AND SYSTEMS TEST THERMAL VACUUM) 
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7. COST EFFECTIVE TESTING 


The issue of test cost effectiveness is a question of value received 
balanced against test costs. For a complex communications spacecraft, the 
total investment in an in-orbit spacecraft can easily exceed $40 million 
(including $20 million for a booster) with an operating revenue on the order 
of $40 million per year. Total recurring unit and systems test costs would 
typically be less than 7 percent of the spacecraft costs, or approximately 
$1. 5 million. Therefore, from a return on investment viewpoint, it is 
apparent that all reasonable test screens should be utilized to ensure in-orbit 
reliability. It also should be noted that the relatively high operating revenues 
create a very significant launch schedule incentive, particularly for the 
initial spacecraft. 

Typical recurring cost data for an acceptance test program are shown 
in Tables 7-1 and 7-2. The data are for programs of moderate complexity 
with 1 flight spacecraft and for 8 flight spacecraft, respecti vely. The data 
are strictly approximations and do not represent actual data for any specific 
"program. However, it will be apparent that large variances are possible 
without significantly affecting the conclusions. With reference to Table 7-1, 
note that the fixed components of unit and systems level acceptance tests are 
67 and 76 percent, respectively. Therefore, the basic test plan is the pre- 
dominate test cost driver. Hardware problems and failures create only 20 to 
10 percent of the total test cost. However, secondary program stretch-out 
costs resulting from systems level problems override all test cost consider- 
ations by a large factor. The schedule implications of the basic test plan are 
also obviously important because of the secondary program costs (typically 
on the order of SZ'iK/day in spacecraft cost and as much as $lI0K/day of in- 
orbit operating revenue). Therefore, based on the cost drivers, it seems 
clear that all practical steps to achieve a near zero-failure condition at 
systems level are warranted. The optimum approach would be zero- failures 
at both unit and systems level through an effective emphasis on defect pre- 
vention at the subunit level. The test effectiveness of the unit level screens 
should also be substantially greater than either the systems test environment 
or the in-orbit environment. In addition, the duration of the acceptance test 
program should be varied in accordance with the actual failure experience 
on the particular set of spacecraft hardware as proposed in section 4.6 of 
tins report. 

For a program with eight flight spacecraft, the relative cost drivers 
shift significantly as shown in Table 7-2. The program stretch-out costs 
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TA8LE 7-1. TYPICAL RECURRING ACCEPTANCE TEST COSTS 
(I Flight Spacecraft] 



Dollars in Thousands 

Item 

Fixed 

Variable 

Total 

8asic unit tests 

200 



Unit rework and retest 
due to unit level problems 


100 


Unit total 

200 (67%) 

100 (33%) 

300 (100%) 

8asic systems test 

750 



Systems retest due to 
problems 


200 


Unit work and retest due 
to systems problems 


35 


Systems total 

750 (76%) 

235 (24%) 

985(100%) 

Program stretch out costs 
due to systems problems 


1700 


Program total 


1700 

1700 

Grand total 

950 

2035 

2985 

Potential savings 



Ratio 




If all problems screened 
at unit level 


1900 


If zero-failures achieved 
in test program 

Maximum permitted 
increase in unit costs 
to achieve zero-failure 
at systems level 


2035 

9.1 


arc not affected, but all other cost elements increase proportionately with 
quantity of spacecraft. The overall effect is to increase the importance of 
test costs. Corrective actions at the subunit level, which result in lower 
failure rates, become more: highly levered. 

Based on an engineering review of the FR data base included in this 
study, it is probable that otuy a small fraction of the failures are random in 
the sense that corrective action is either impossible or not cost effective. 

In fact, there are many corrective action paths which do not involve increased 
manpower. For example, a recent Hughes experiment in an electronics pro- 
duction area indicated that the manufacturing defect rate could be dramatically 
reduced by simply assuring rapid feedback of problems tc the person res- 
ponsible, It is probable that an aggressive corrective action program to 
achieve near-v.ero failures in both the qualification and acceptance test pro- 
grams would bo cost effective. 
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TABLE 7-2. TYPICAL RECURRING ACCEPTANCE TEST COSTS 
{8 Flight Spacecraft) 



Dollars in Thousands 






Fixed 

Variable 

Total 

Basic unit tests 

1600 



Unit rework and retest 
due-to unit level problems 


800 


Unit total 

1600(67%) 

800 (33%) 

2400 (100%) 

Basic systems test 

6000 



Systems retest due to 
problems 


1600 


Unit rework and retest 
due to systems problems 


280 


Systems total 

6000 (76%) 

1880 (24%) 

7880 (100%) 

Program stretch-out costs 
due to systems problems 


1700 


Program total 


1700 

1700 

Grand total 

1600 

4380 

1 1 ,980 



A$ 

Ratio 

Potential savings 




If all problems 
screened at unit level 


3300 


If zero failures achieved 
in test program 


4380 


Maximum permitted 
increase in unit costs 
to achieve zero-failures 
at systems level 



2:1 


Relative to qualification tests, the results discussed in section 4. 5 
of this report indicate that none of the unit or systems environments are 
very effective as a screen for design problems. The data suggests that an 
increased emphasis on worst-case analysis, development tests, anti other 
nonrecurring activities would be cost effective, including an aggressive cor- 
rective action program, intended to achieve near-aero failures in the unit 
and systems qualification test programs. With such an approach, the need for 
a qualification spacecraft becomes dubious, It is probable that destructive 
tests arc not required to achieve an effective screen and there is no apparent 
schedule advantage to a sequenced development. Therefore, a protoflight 
spacecraft is probably the most cost effective path wherein the first flight 
spacecraft is exposed to a mare comprehensive test program. 
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In summary, a cost effective test program would most likely have the 
following attributes: 

1) An aggressive corrective action program would be pursued to 
achieve near-zero failures in both qualification and acceptance 
test programs. Such a program would include standards regard- 
ing acceptable defect rates in various categories (e.g., design, 
workmanship, etc. ). 

2) The effectiveness of the various test screens would be measured 
and an aggressive policy would be pursued to maximize the 
effectiveness of the overall test program. The key is to ensure 
a high level of novelty in each test period. Time alone is prob- 
ably not an effective screen in any given stable environment. 

i) In particular, the effectiveness of the unit test environments 
would be increased significantly to ensure a near-zero failure 
rate at systems level. 

4) Observed failure rates would be used to predict both systems 

level test results and in-orbit performance. The content of the 
test program would be varied in accordance with these predictions. 




8. CONCLUSIONS 


The test programs considered by this study were extremal,, „rf *• 
in ensuring a high level of in-orbit reliability. Only I t CtlVe 

problem occurred during almost 44 n ^ le catastr °phic 

soacecraft tv,,- „ , , g aimost 44 years of in-orbit operation on 12 flight 

I,...,. The i e f U ! tS of this stud V indicate that in-orbit failure rates are 
highly correlated with failure rates observed in the unit a !S Z l * 
programs. Further, the data suggest that *• d 3ystems test 

p r o fi r , m To ‘,1 h'io itTnilT S " SSC ( St . thi “ an *8grc ss ive corrective .cion 

u 52:322: x*r?* 

gggS^^SSHSSSs 

8. 1 GENERAL 

1) Normalization of data bases from similar programs is feasible- 
and "> hen based on electronic parts count, number of units -ml' 

P-^m1auf erem tm ‘ S ' Pr ° vW - program, u, 


2 ) 


3) 


Approximate iy 1 FR of any typo was generated against each 
127 electronic parts produced on each of the three programs. 

Only 68 percent of the 1-Us were primary, 20 percent were 

On*Tlfc"r F« r , tC ri (IK r> indlU L ‘ d ' and 11 P^^T-nt were secondary 
Of tiu In 1 1 os, 41 percent won electrical test errors and ' 

24 percent were electrical over stress . I tv T Fits remained 

essentially constant as programs matured, and eventually ul„,ov 
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equalled the primary FRs generated on later spacecraft at the 
systems level. Increased emphasis on test procedures and safety 
practices would have been cost effective in reducing the I&tT FRs. 
Of secondary FRs, 64 percent were for acceptable out-of- 
specification test results. Again, better engineered procedures 
would have reduced the FR rates. 

4) Z and X were more effective than Y in vibration testing, 

and the eclipse simulation was the most significant phase of the 
systems level thermal vacuum test (48 percent of all thermal 
vacuum FRs). 

5) The ambient initial checkout and performance tests are very 
effective with at least 42 percent of all unit and 48 percent of all 
systems level FRs being detected at these test phases. 

6) As the programs matured, the FRs generated at both unit and 
systems level declined. The first spacecraft has three times as 
many FRs as the average of all the subsequent spacecraft. Addi- 
tionally, the last spacecraft in any program set has an increased 
number of FRs compared with the preceding spacecraft because 
of delays which have resulted from unit problems. 

7) Workmanship problems dominate the primary FRs (40 percent 

of total and 51 percent of acceptance). Sixty percent of all work- 
manship FRs are installation and/or assembly errors. Tittle 
evidence of learning in this area was evident. An emphasis 
placed on reduction of instalLation/as sembly problems should 
result in substantially reduced FR rates, 

8) Part problems remained essentially constant throughout the test 
programs at 22. S percent nf all FRs. Forty four I'ircent of the 
qualification and thirty one percent of acceptance part problems 
were caused by manufacturing deficiencies . 

9) Subsystems with high concentrations of electronics (Tn-C and ACS) 
had a disproportionately high number of Fits. Power and propul- 
sion subsystems had a disproportionately high lumber of It.- 1 Fits. 

10) Half of all in-orbit problems were duplications of previous occur- 
rences, Fifty-two percent of the unique problems were design, 
the majority of which could not have been detected by the accept- 
ance or qualification lest programs ar designed. 

8.2 CUAI.IFICATION TESTING 

The unit and systems Level qualification tests we i j found to be 
marginally effective in delecting design deficiencies. Approxi- 
mately 8,4 percent were screened it unit level and 22, ) percent 
at systems level, for an overall effectiveness of 28,8 percent. 


1 ) 


These results suggest not only that the test technique, shoold he 
improved, but also that greater emphasis should be placed on 
subunit development tests, worst-case analysis, design review 
techniques, etc. 

2) Systems level vibration testing is an effective qualification test. 

3) In general, systems level tests are more effective for qualifica- 
tion purposes than for acceptance purposes. 

8.3 ACCEPTANCE TESTING 

1) Most vibration related problems are detected during qualification 

’ tests or unit level acceptance tests. Thre.-.x.., systems level 

vibration testing is probably not cost effective. 

2) All systems level acceptance thermal tests were only 8 percent 
effective. This is probably the result of deficienc.cs in test 
technique and not strongly related to total test time in the environ, 
ment. q Virtually all of the failures were detected during the first 
60 hours of thermal vacuum testing. The effectiveness of .he 
system thermal environments may be severely constrained by the 
inherent limits imposed on temperature ranges and rates of 
change. 

3) Overall the acceptance test programs wore highly ^ctive in 
screening critical defects (99 percent). For all problems, the 
combined 8 unit and systems test effectiveness was 87 percent f„. 
Program 1 and 7 3 percent for Programs 2 and i. 

8.4 RELIABILITY GROWTH 

1) Test effectiveness models can be used at both unit and systems 

’ level to vstim.L- the population oi defeats and to prcd.c l~rto - 
man cp in systems test and in-orbit. In-orb.t failure - ate* a,< 
strongly correlated with test program results. Ihose * 

could be used to vary the content of the tost program m arm, duu, 

with reliability requirements. 

2) The in-orbit cumulative failure rate data exhibits a consistent 
slope which implies a constant equivalent lest effectiveness* 

\) The in-orbit failure rates generally decrease \viUi successiv.' 

members of a spacecraft series, indicating significant r. lu i 3 
gruwth. The cumulafve failure rate at the end ..I sy.m-ms Ips. 
is a good predictor m in-orbit performance. 


t 


rr 


■r 
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be a c/u ratelVcombined into a sil^U^D ^ ^V*'® rate data cannot 

are generally different. The disnllit^hT' 6 ^ ^ Gcause slopes 
erally decreases with aun c^i, P y beuveen the slopes gen- 
For a mature design (e - ° f a ^Pacecraft^eries. 

able that a single Duane curve could b^tfri* I0)> it: is P rob ~ 
purposes. could be utilized for extrapolation 


COST EFFECTIVENESS 


1 ) 


Z) 


4 ) 


great to permit significantly reduced t eLin 1/^ 8enerall V' ^ 
program should be considered a mafj « 8 ‘ H °"' cver > the test 
technique to achieve reliability growth. * maurance and not a 

schedule del* y e ,he uT.«Xl^Sw. 

A continuing effort is wai-canf^ . 
sceeedid, of each 

definition oUhJteTt program! 11 U ' St C ° StSi are by the 




APPENDIX A, SUMMARY OF INFORMATION CODES 


(A DESCRIPTION OF INFORMATION 
OR CODES ENTERED IN 
EACH COLUMN) 


Column 


1 through 5 


7 


Description 

FR or j£R number entered here. Columns 
2 and 2 contain a non-U, S. subcontractor code 
of IFRs only. 

Level - Indicates reporting level where 
failure occurred. A code is used to indicate 
these levels: 


8 


9 


N 

F 

P 


0 

1 

2 

3 

4 

5 

6 
t 

H 

9 


1 

2 

6 

/ 

K 

9 


bnit (foreign or domestic) 

Spacecraft (systems test) 

Post-launch (orbit) 

Subsystem - Spacecraft subsystem from 
which part failed, coded: 

Other (thermal, etc, ) 

Communications 

Antenna 

Power 

Despin control subsystem 
T&-C digital 
TfcC RF 

kc:s 

St ructure / mechanical 
Harness assembly 

Activity Type - During which failure 
occurred, coded: 

Acceptance tests (any) 

Qualification tests (any) 

Alignment 
1 .auric h ope rations 
I .nunc h /!» hi s t 
Orbit 


A -I 
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Column 

Code 

Description 

10 


Test Phase — In process when failure 
occurred, coded: 


t 


12 through 14 
16 through 1 8 


19 


20 


20 


0 Inspection 

1 Electrical manufacturing 

2 Mechanical manufacturing 

3 Electrical performance test (first, not- 

defined, or last) 

4 1ST (first or last) 

5 ESTV/DCTV (electrical shelf or despun 

compartment thermal vacuum) 

6 Vibration (sine or random) 

7 TV/STV (spacecraft (or solar) thermal 

vacuum) 

8 Thermal cycle 

9 RCS tests 

Unit Part Number (P/N) — Of failed item. 

Part P/N — Of failed component, if 
applicable. 

Importance Corle: 

- Primary type of failure/anomaly 

T Test or integration induced 

S Secondary problem — no action taken 

except NCMR or TER close out 
A Anomaly not verified w /retest 

D Duplic ate of in-orbit problem 

Type code for test (N integration) induced: 

1 Damaged wires, heaters, and other leads 

2 Electrical test overstress error 

3 Electrical test error 

4 Hardware handling problems 

5 Wrong environment imposed 

(> Had part selection in test 

7 RCS test error 

8 Test fixture induced 

9 Spacecraft operator error; in-orbit 

Type code for -ccondary problem: 

0 Out -of- spe*. /OK 

| Measurement technique bad /no retest 

2 Test equipment error/no retest 
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Column 


20 

(continued) 


21 and 22 
23 and 24 
25 

27 through 5 1 


53 


54 


Code 


3 

4 

5 

6 


1 

2 

3 

4 

5 

6 
7 


0 

1 

2 

3 

4 

5 

6 
7 
H 
9 


0 

1 

2 
4 


Description 

Manufacturing problem or assembly 
error multiple duplications 
lest error /no rework 
Miscellaneous/no rework 
Anomalies /no retest 

Month during which failure occurred. 

Daji of month failure occurred. 

Last digit of Year in which failure occurred. 

^{erba l complement to failure code . 

“ “^7 a ' omple,e o t ^ ,T. P 

cause between this verbiage and failure 
code of Column 62 and 63. 

Symptoms - Of failure, coded: 

Anomaiy or specification variance 

random occurrence 

Periodic / inte rmittent 

Hard failure 

Normal life or wearout 

Hot hardware 

VisuaL observation 

— vcrit y - ° £ failure, coded: 

Hot critical 

Rework l u existing paper 
Retest to existing paper 

Degradation of performance or mission 

Catastrophic failure 

Redesign hardware or paper 

Replace failed part 

Hot hardware 

Hot verified 

Operationally overt omc 
Genei-it 1 , pe - Of failure, t oded: 

1 nknown/uin-e rified 

^ o rkm< inship 

Part failure 

Design or performance 

Ho tail 


A - < 


I 


Column 


57 


57 


57 


59 


Code 


0 

1 

2 

3 

4 

5 

6 

7 

8 
9 


0 

1 

2 

3 

4 


0 

1 

2 

3 

4 

5 

6 
7 
H 


0 

1 

2 

3 

4 

5 
(■ 
i 

M 

9 


De sc ription 

Type code for Workmanship failure: 

Installation /as sembly 

Test error 
Paper error 

Bad electrical connection 
Accidental handling 
Misuse 

Wrong part used 
Damaged wires /coax 
Contamination/ corrosion 
Bad alignment or mac Inning 

Type code for Part failure; 

Unknown 
Workmanship 
Contamination 
Manufacturing process 
Design 


1 ype code for Design failure: 

No change 
W rong part 
EMI 

Physical properties 
Specification error 
Process error 
Circuit design 
Mechanical installation 
Analytical error 

Lil' ■ nn icntal Condition ~ Attributed to or 
cxis.ing at til ue :>t failure. 

Ot he r 'orbit 

Ambient 

U.imloi 1 1 

Sim- 

Hot 

Cold 

Eclipse (simulation) 

The rmu I / trims ition / pumpdnwn 
EM I /EMC and stain change 
k addition 














Column 


Code 


Description 


jest_Modif) er _~ F or activity, coded: 


1 Initial 

* Final 

X, Y, Z Axis (X, Y, Z) 

B Be fo re 

D Durir.g 

A After 

Numerical portion of Failure Code 
(cause of failure). 

Unknown (failure vfiri K*<v- 

02 In process fiedi but caase unknown) 

03 Contamination (e„ g. , weld splatter etc 

ru in electronic part or RCS) ’ ' 

Failure not verified (failure of part not 

verified by test/analysis) P 

il Incorrect specification 

Suboxide arc-over 
D Handling 

!q J. est : rror (operator error} 

Manufacturing error (a mistake) 
j Design inadequate 

Flectrical overstress 
Z Micro cracks 

Cracked chip 
Thermal overstress 
Component (internal) short 

I WrZrS.lu UnClea ' " 

- op„, or active weW or 80lder 

^ Normal and wearout 

I Inte metallic formation 

! Hilh e !d l n V / and unn f ces ^ry test conditions 

High window impedance 

Mechanical tolerance 

Component (internal) open, missing, or 

defective connection 

Test equipment malfunction/design 

Manufacturing process 

Minor RF anomaly 

Paperwork lags spacecraft hardware 

^“ble 9PeCi£iCati ™ c °odition- analysis 

Hermetic seal not sufficient 

Performance specification too tight 

SlKZ?° U ’ ' aii “ r ' h ‘ ,t0r V " ■'“"rccurrine 
isolated occurrence ® 


I ^ 
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Column 


Code 


Description 


62 and 63 
(continued) 


64 


66 through 68 
70 

72 through 74 


75 


33 Miswiring 

34 Design subject to environmental parameter 
drift 

35 Test equipment or structure unsafe or 

subject to human error 

36 Part lost — no analysis 

37 Inspection error (a mistake) 

38 Checking/inspection insufficient or 

non-existent 

39 Design/procedure subject to human error 

40 Premature wearout 

Letter Portion of Failure Cod e (corrective 
action) 

P Primary part failure (part caused), Part = 

Hi Rel in FPS01000. Basic corrective 
action involved a part supplier. 

S Secondary failure, induced by external 

error such as poor workmanship or 
design. 

Unit Serial Number - Of failed item. 

S/N of Spacecraft upon which unit "flew" 

Time D ur ation represented as XX. X 
(vvho'-e ’table) 

Time Base (as applicable): 


77 through 78 


80 


0 Seconds 

1 Hours 

2 Months 

3 Years 

4 Days 

Test Cycle, Retest Number (R-), or 
multiple of similar failure. 

Program Data Set 

0 Program 1 

1 Program 2 

2 Program 3 
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APPENDIX B. ANALYSIS OF SOME SIMPLE 
FAILURE RATE MODELS 


SUMMARY AND CONCLUSIONS 


Duane d *“ * PP “” '° fit so wel1 *“ 

least an inhiiHva _ a " analytical model based on at 



an mod * 1 . bas « d « at 

the applicability of several such models Th * ?pe '’ d ‘ X investigates 


an . d approaching aero aft 

for^Sf^ odelctearVy’ /oe’’ ' Iff ‘^T b ™ aa 

sirapliatic model is rejected * Dua "' S,tl '« bt h ” c a " d - therefore, this 


first rjde! * «*?? anphistication of the 

of the cumulative failure rSeV„^‘d^ r V™VLm ° g d P ? P *'': l*" sl ° pe 
range observed in actual ground test and in-orbit faUmo rato^daU 


probabiU^ t di^tribu^/ons da va l ryi ) ng^es , t V eKect^vity , eJY™ *“° C> /Section 

^ STS*, 


model based on an intuitive understanding of the test/failure process. The 
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key element of this model is the distribution of the defect population in terms 
of probability of {or vulnerability to) detection. Further development of the 
model can provide more complete correspondence with trends seen in the 
empirical failure rate data, 


EQUAL PROBABILITY OF DETECTION FAILURE RATE MODEL 

The first model considered is similar to the one investigated by 
Timmins*. Here we assume the existence of a fixed initial population of 
defects which is steadily reduced by successive test screens. The defects 
all have equal probability of detection, and the test screens all have equal 
effectiveness. We further assume that the failures which occur in a test 
screen (interval) arc a constant fraction of the rcm.iining defects, (These 
constraints on the model can be removed later, but are imposed to keep this 
first illustrative example as simple as possible). Let N 0 - the number of 
defects in the initial population 

= the number of remaining defects after k test screens (or 
intervals) 

F^ - total failures (defects detected) after k test intervals. 



T = time interval of each test screen 

a = the fraction of the remaining defects which are detected in each 
test interval. 

X*. = cumulative failure rate 
k 



kT 


In acco 'dance with the above assumptions, the number of defects 
remaining after (k + 1 ) test intervals is given by. 




Timmins, A, F. , "A Study of Relationship Between Performance in 
Systems Tests and Space", Prnceedinus of the Institute of Environmental 
Science s ■ 1975, 1.172 
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The solution of this difference equation is a 
decrease of the defect papulation. 


simple 


geometric 


progr ea sion 


N k =I V -a) k 


which produces the following relations for F, and 

k 


= N o [l . (1 - a| k ] 


a N f 

o ] 

T 

m 


L^(l - a)* 


ak 


(U 


Figure B- 3 shows a plot of Equation 1 on log-log paper for two differ 

tivitV lU< F ° f th ® P arameter a (which corresponds physically to test effec- 
tivity, From this plot it is clear that the model does not fit a Duane straisht 

niti'allv 3 l nd ; c r ed in the figure ' the im P lied rate is nLrly consUnf 

lmtiaHy, and then approaches zero after many teat intervals. The deficiency 

th 1 assu , m P tlon of ec S ua l probability of defect detection, which * 
results in a relatively rapid initial depletion of the defect onnulaHnn ( n , 
by a diminishing trickle of failures from the Seated ^populaHon ' 


follows: 


- » i • 

; A continuous-time version of the same model can be developed as 


N q = the initial number of defects 

N(t) = the number of remaining defects after t hours 

F{t) = N q - N(t) = total failures over the interval (0, t). 

Then in accordance with the assumption that the failure rate is proportional 
to the number of remaining defects, } 1 


N (t) = - 


N(t) 


where, t = test effectivity time constant 
effectivity) 


(a parameter characterizing test 


I 
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Then, 


N(t) = N e' t/T 
o 


F(t) = N (1 - 


e ' t/T ) 


and the cumulative failure rate 


is expressed as, 


Mt) 


Fit] , No [l - e' t/T 

1 T L~/“) “ 


( 2 ) 
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Decrease of Defect Population with Test Time 

This approach is pursued mathematically by defining a defect density 
function, D(v, t), which is both a function of time (since the defect population 
decreases with test time) and a function of the defect vulnerability to detec- 
tion, v. In this illustrative case we consider the simple density function 
shown in Figure B-3, Here the defects within the population are distributed 
in terms of vulnerability to detection between the values v = 0 (impossible to 
detect) and v = 1 (easy to detect). In this example we also assume that the 
initial density is uniform; that is, the initial defect population has equal 
density at all values of detection vulnerability. (Further shaping of the result 
ing failure rate curve is accomplished by introducing a different shape for 
the initial density function, e. g. , a Gaussian type curve). 

We can therefore make the following definitions: 

v = the relative vulnerability of a defect to being detected in test 

(resulting in a failure) 

D(v, t) = the defect density function, which, for any instant t, gives 

the number of defects having test vulnerability between v and 
v + Av 

D(v, 0) = N c = the initial defect density (assumed constant in this 
example) at t = 0 


N(t) = the total number of defects remaining in ihe population 


At any instant the total number of defects remaining, N(t), is found by 
integrating the density function D(v,t) over the range v = 0 to v = 1, i. e. 


N(t) 



D(v, t) dv 


where, for \ = 0, 


n(G, = 



N dv = N 
o o 


(3) 


is the total number of defects initially. 

The decrease in the defect population (as a result of test action) is 
obtained from the differential equation, 


D(v, t) - D(v, t) 


(4) 


f - 
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where, 

T = test effectiveness time -constant (a parameter of the test process). 

Equation 4 states that the rate of decrease of the defect population at a parti- 
cular vulnerability level, v (i. e. , the population contained in the vulnerability 
range of v tc v +Av) is proportional to the test vulnerability of the defects as 
well as the number of defects remaining. The solution to Equation 4, giving 
the density as a function of time is then, ® 


t_ 

D(v, t) = D(v, 0) . e" v t 


(5) 


and, since for the particular case being considered, D(v, 0) - N 

o 

D( v, t) = N . e" V T 
o 


{(>) 


Figure B-3 shows a plot of Equation 6 for several values of t, illus- 
trating the resulting decrease of the defect population with test time As 
indicated, the population is depleted rapidly in the region of highest vulner- 
ability, and more slowly elsewhere, according to the exponential relation 
(Equation 6). 

The effect of a change in test effectivity can be accounted for by- 
introducing one more parameter, (3 , defined as follows: 

P= relative test effectivity 

Thus, in the present case we assume a ground test period of duration t 3r 
followed by an in-orbit operation period. The relative test effectiveness 
of the in-orbit operation is taken to be (3 times the effectiveness of ground 

test The differential equation describing defect population decrease in orbit 
is then. 


1 

I 

I 


D(v, t) = - P • • D(v, t) (for t > t ) 

which has the solution, 

Iq t / ^ V 

D(v, t) = N q e" V “ . e- V ^ P ("T - 1 j (for t > t Q ) 


(7) 
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above is formed as follows: 

N(t ) = total number of remaining defects 


whe r e , 


N(t) = 
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D(v, t) dv 


F(t) 


= N . N(t) = total failures ove 


r the time interval (0. t) 


x (t) = = cumulative failure rate 
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( 10 ) 
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relative to the character of the failure rate curve and the decree of its 
Duane fit. 


1) Ground Teat Period . Over the ground tent period the average 
log-log slope of the curve is roughly -0. 4, wh ic h -is—i n-the— ra-n-ge — 
of values typically used with the Duane model. Thus, the ground 
test curve can be fit reasonably well with a Duane curve (straight 
line on log-log paper) having a slope of -0. 4. Further, 
"straightening" of the curve can be achieved by starting with a 
different initial defect distribution function — one that is skewed 
toward lower vulnerability rather than the simple uniform ' litial 
distribution used in this example. 

2) In-Orbit Period. The portion of the curve corresponding to 
in-orbit operation has a slope which varies from. -0. 77 to -0. 95 
over the first decade. The average slope is -0. 8t. over the 
decade, and the curve can be fit very closely by a straight line 
of that slope. This eagrecs reasonably well with actual in-orbit 
failure rate data (c. g. , Figure 5-20) for which the average slope 
for the F-l through F-8 data is approximately -0.75. 

Mor cover, the transition from ground test to in-orbit operation, 
which shows up clearly in Figure 3-4 as a change in slope, can 
also be seen in the actual data. For example, the F-3 data 
shown previously in Figure 5-4 shows a similar slope change 
at launch. 
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